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ABSTRACT 

The majority of Latin American immigrant students in 
ESEA Title VII Spanish/English programs that have been evaluated seem 
to suffer from inexperience in taking tests like the .Comprehensive 
Tests of Basic Skills (CTBS). This study was designed $o assess test 
familiarity, language of the^test, and literacy, and to examine 

9 whether certain alterations in the testing procedure might allow for 
better estimation of students 1 ability and program benefits. Limited / 
English proficiency students in 'grades four, six, and eight' 
participated in the study Alnst rumen ts used were the English and 
Spanish versions the CTB5S. Testwiseness training was developed and 
administered to hlMf the groups. An oral administration procedure for 
students wifch limited native language . literacy skills as well as 
limited English proficiency was also developed. A socidlinguistic 

* analysis and a psychometric, analysis of the tSsts were developed and 
testwiseness 'materials were designed. The £i'story bf the development 
of this prdcedures, 3S % well~as the try outs and revision?, a 
description ot-the teacher training, £nd a detailed analysis of the 
testing data comprise the /greater part of the report. Attached to the 
study is the English 1 version of a teacher '.s manual for training ^fn 
testwiseness, a program adapted^for Hise^with the. CTBS. (AMH) 



j 



v Vr 



**************************************** 

.* Reproductions^ supplied by EDRS are the best that can pe made * 

* from the origi na^ document . . tl * 

*********************************************************************** 



sr, IMPROVING ESTIMATES OF ACHIEVEMENT IN 
STUDENTS WIT^> LIMITED ENGLISH ^PROFICIENCY 



A S£udy Performed Under a Grant Award 
from, the National Institute of Education 
Grant Number NIE-G-80-0081 



' Principal Investigator 
Richard M. Pip^r 

* ' * Investigators 

William J* Doherty 
Nancy A. Russo 



Applied Social Research, Inc. 
15219 Sunset Boulevard, Suite 201 
Pacific Palisades,. California 90272 
' ' (213) 454-8464 



U* DCPAHTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 
EDUCATIONAL RESOURCES INFORMATION* • 
CENTER (ERIC) 
»X Th» document has been reproduced, a* 
received from, the person or organization- 
originating rt, 
□ Minor change^ have been made to improve 
reproduction quality. 

• Point* of view or opinions atated in tha docu- 
ment do not naceaaenry represent official NIE> , 
portion of policy, 4t5F' 



I0f 



V 



TABLE OF CONTENTS . 

'* ' Page 

1. The Problem v 1 

2. Design of the Study 2 

3. Procedures • •* • r . . . . • 4 

4. Results 24 

A. The linguistic analysis. . 26 

# . ' _B> An analysis of effects . .* 59 

5. • Discussion and conclusions • • 72 

6. Appendix: "Teacher's Manual for Training in 

Testwiseness " ..•..(•• 81 



/ 



, ACKNOWLEDGEMENTS 

The authors wish to express "their appreciation for inval 
uable 'assistance in the conduct of this research. Thanks go 

to: • • 



(1) Belia, Cruz, Arden Daniels , 'Sabiniano 
Legarreta, Tom Martin, Alice Petrossian, 
Dr. Martin Pilgreen, Dr. Marie Plakos, 
Elsa Santillan, Mary Skandera, and 

• Lilia Stapleton fc^ permission to con- 
v ciuct the study in their respective dis- 
tricts and schools. 

(2) LintfSi Calderon, Mary Calderwood, 
Graciela Carmona, Hector Carrio, Suanna 
Ponce Gilman, , "Barbara Harper*, and Maria 
Otero for coordinating the work in # their 
respective . districts, conducting test-/ 
wiseness . training and testing. 

. v 

(3) Dr. Lois Weinberg for writing the Eng- 
lish version of the testwisen.Sss train- 
zing materials/* * 

(4) Dr. Joe Lucero for -designing Ind assist- 
ing in the 'socio-Iingui^tic analysis of 
the CTBS. t ' , ■ ; 

(5) Tony Brito-, Dr. Victor Rodriguez', and 
jbinda Smith for recording €he CTBS items , 



/ . 



.0/0: 



THE PROBLEM 



Sevel^l years ago, the Principal Investigator (P.I.) 
served as"~a\teacher in a school in Nicaragua. ^The school , 
as required oy law, was organized according to* the French 
education, model . Thus, instruction was highly oriented to- 
ward the development of ac4demic skills. .High performance 
was required. Those who could not me^t the standards Either 
dropped- out or flunked out. The few who made it into and > 
through the. bachillerato program were the intellectual cream 
of the country. ■ „ 

Sinpe many of- these bachi^leres would be going to col- 
lege in the United States, the P7^_became interested in 
determining their academic status vis a vis high school grad- 
uates in the Uoited States." Permission a was obtained to give 
them the Spanish v^rSion of the Differential Aptitude Test 
(DAT) battery. The expectation was that A based on their per- 
formance in school, they would obtain scores in v the upper 
ranges of the U.S. norm distribution. This expectation was 
not fulfilled. Ih fact the students did quite poorly. This 
unexpected result led the P.I. to conduct a discussion with 
the students. One £hing that seems -to have, bothered them 
was the format of the test. It was different than ahy thing 
they had experienced in their educational careers; As a con- 
sequence, they had never developed ttae test taking ■ skills 
relevant specifically to taking objective norm-referenced 
tests like the DAT. 

Since returning to the United States, the P. I; has 
served as the evaluator of a number of ESEA Title VII Span- 
ish/English bilingual programs. The majority of Latin Ameri- 
can immigrant students in the programs', evaluated seem to 
suffer from t^ie same inexperience in takjlng tests like the • 
Comprehensive TeJre of Basic Skills (CTBS) . • If this is true, 
then in all probability, thg< test scores they obtain under- 
estimate their true levels of. performance. * 

There may be other problems ,in using* tests like the CTBS 
to measure the educational achievement of language minority 
students. One of these, is the language of the test. Prior 
to the introduction^ of the CTBS (Espahol, many of these stu- 
dents were examined using the CTBS, Form S XCTBS/S) . The 
absence of a match between the language of the test and the 
language, of the examinee as well as tjie absence of a match 
^ejwesen the oral/aural ^language ta£ks addressed, in English 
as^. Second Laxiguage (ESL) k programs and the language tasks 
addressed in the*£TBS/S motivated recommendations of caution 
.^ client school districts regarding the use of -the* test with 
"language minority students arid, when used,, the interpretation 
o£ scores. It was felt that such scores systematically under 
estimated both student conceptual skills and-prpgram benefits 



Given these feelings Qf discprafort with the CTBS/S, the 
introduction 'of the CTBS Espanol was welcomed. This version, 
which was A largely- a translation of the CTBS/S (some Unique 
Spanish language material was introduced ' for linguistic, dnd 
cultural reasons) , began to be used by some districts with, 
some Spanish dominant students. The results of this shift , 
were predictable. These students, depending on their levels 
of primary language literacy, regularly obtained higher scores 
on, the reading, language and mathematics concepts and applica-> 
•tions subtests than they had previously obtained on the paral- 
lel English language version. The CTBS Espanol offered a 
means* for estimating student conceptual skills as well as the 
benefits of instruction in Spanish reading *and language arts; 

Questions still remained. Many practitioners questioned 
the validity of a Spanish-rlanguaga test that was /fnerely a 
translation of an English-Slanguage test. To what degree would 
such 'a. test "reflect non-Spanish.' modes of expression?. To what 
degree would' it be cultural! v biased? Further, what inferences 
could be itfade based on item scores? If a student failed an 
item, could thds ,be t attributed merely to an Absence of- lit- \ 
eracJy or was there, beyond £ possible absence of literacy, 
a "lack of knowledge of 'the* concepts represented by the wdrds? 

Out of these observations and- questions, the problem 
addressed by this study was formulated. The. major assump- - . 
tion underlying the study is that the' factorial composition 
of tests like the CTBS/S is different for students with 
limited English proficiency (LEP) than for students with full 
English proficiency (FEP) . This f actforial, cgmplexity intro- 
duces systematic error into estimates of LEP students' ability' 
because <the score is not a pure measure of .the ^ability . The 
sources "bf this factorial complexity may- be many. Thi's study 
focuses on three: unfamiliarity with the test format, lit- 
eracy, and the language % of the test. 

The study,* then, was designed to directly assess test 
familiarity, language of the test, and literacy % and/to exam- ' 
ine whether certain* alternations in the testing procedure 
might allow for better estimation of students' ability and , 
program benefits* 



DESIGN OF THE STUDY 



A three-^factor factorial design "was *u^ed. The factprs-* 
were • (1) # language jDf the test (Spanish, English), (2^ -Admin- * 
istration Conditions ( standard , or al ) and* (3) t?st familiar- 
ity (trained,, untrained-) * By crossing these factors, eight- 
treatment groups, -were obtained as follows: ' 

' 1. Spanish version, of the- test, stan- 

dard* administration, testwiseness 
m * . -straining. > * ' 9 

s • ' ' , ' - 



2. Spanish: version of the test, stan- 
dard administratibn, no testwiseness 
training. 

3. Spanish version of the test, oral 
administration , testwiseness train- 
ing. „ „ 

4. Spanish version* of the test, -oral 
administration,- no * testwiseness^ 
training. 

5. English version of the test, stan- 
dard administration, testwiseness 
training. * r 

6. English version of the test, stan*- 
dard administration, no testwiseness 
training. 

7. English version of the. test, oral 
administration^ testwiseness train- , 
ing. . 

8. English version. of the test, oral^ 
administration, no te;stwis s eness 
training. 



Students who participated, in the study came from grades 
four, six, and eight. All had Spanish as their primary lan>- 
jjuage and were .limited English proficient. In order to con- 
trol for* factors that might* bias ^the result, they were as-r 
sighed to conditions on a modified matching' basis in which 
controls were institute^ for country of birth, amount of prior 
education, socio-economic status, length- of. residence in the 
U.S'. ,. grade level", and entering' ievels of English and Spanish 
language proficiency. 

Instruments used were the CTBS, Form S and the CTBS 
Espanol, Level 1 (with fourth grade participants) and Level 
3 , (with sixth and eighth grade participants). The lattery- 
was a translation of .the* former completed by staff in the 
•Norwalk-La Mirada (CA) Unified School District. In some 
instances >' material that was considered inappropriate for 
translation was replaced by more culturally and/or linguis- 
tically * appropriate material.* % 

The testwiseness training that was^ administered, to half 
the groups was * developed -as a part of this study. Its char- 
acteristics % are described later* in this report. A ^ 

The oral administration prqcedure was* also developed a's 
£ part of this study. Its characteristics $re also described 
later in this report. * j 

* • 
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PROCEDURES 



Linguistic Analysis of /the CTBS 



It is frequently claimed that norm-referenced tests used 
for measuring the aptitude and achievement* of students in the. 
United States are culturally biased^ The ^claim' is accepted 
.as true by a large number of educators and, in response" to* 
the claim, ar variety of attempts have been made to create 
culture- free or culture-fair -testis. * Indeed, any manufac- 
turer who produces and markets a test 4 for broad use must 
demonstrate that some effort to rid the instrument of cul- 
ture bias has been made. 

Such v is the case with the manufacturer of the CTBS.^ 
They define bias as the systematic production of upfair 
scores for a particular group. In the technical manual, the 
manufacturer carefully ^desqribes the itenTtryout and evalua- 
tion procedure with Samples of x BlacJC, Spanish- speaking,^ 
"other," and "standard" students, 'In the judgment of the 
writers, reasonable care was takerf^ to eliminate bias as de- 
fined. - + 

Nevertheless, for all the care that may be taken, it is 
impossible- t9 remove all bias from a test. This follows from 
the fact that item writer (s) write from within the framework 
Of their own culture (s). They can do no other. There is no 
such thing as a culture-jfree person or a culture- free position 
This being true, any tfest will be unbiased for some popula- 
tions and biased for others. We cannot remove bias from a 
given test for "all prospective populations of examinees. We 
can only replace one bias witlf' some other bias. % Given,, there- 
fore, that .bias will inevitability be present, our task is .to 
try to reduce its effects for those sub-populations that are 
negatively affected. ".Our first step in this direction was- & 
comparative sccio-4-inguistic 'analysis;, of the English and Span- 
ish versions of the CTBS. 



.Design. of the Linguistic" Analysis 

0 

Two analyses were designed,, a socio-liriguistic analysis 
and a psychometric analysis. The socio-linguistic analysis 

'was designed to facilitate a direct comparison 'of the English 
and Spanish versions as well as to inform £he interpretation 
of our item analysis. . Accordingly, parallel English and 
Spanish versions of items from the Reading Vocabulary, Read- 
ing • Comprehend ipn, ^Mathematics • Computation,, and Mathematics 
Cpjicepts and* Applications subtests were pas'tedon the face 
of 5X8 index cards. *0n the back of each of these cairds was 

^pasted the analytic outline pteSented- in Exhibit 1. 



► % 
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' . EXHIBIT 1 

> 

OUTLINE FOR CONDUCTING THE 
' SOCIO-LINGUISTIC ITEM ANALYSIS 



Phonological interferences ; 
Spani sh ' 



English * 



Lexicon; iifterferences, translations, high and low frequency items : 



Spanish_ 
English 



Grammatical structures : 

Spa ni sh 

English 



< 



Visual cues/stimuli used with test items : 
■ — - 

Spanis h * 

English g 



Socio-linguistic variables : 

Hispanic/Spanish dominant other 
than Mexican American 



Hispanic/Mexican American 
English dominant 

Angl o y\ 



v 



Hispanic cither than 
Mexican American 



Mexican American 



Additional Comments: 




"W6 9 



.The psychometric analysis was designed to focus on item 
difficulty, item discrimination, and item response frequencies. 

f s 

Conduct of the Linguistic- Analysis 

• The 'item cards described above were ' given- to "a Spanish/ 
English. bilingual linguist. .He conducted aji analysis of each 
Spanish/English -item pair for Levels B, C,'l'„-2, and 3 of 
the CTBS. His comments were used as a basis for Interpreting 
the' results obtained in the item analysis-.' ' . 

The psychometric analysis focused primarily, on the 'indi- 
vidual items. Item difficulty was computed as percent' of 
correct response. This was the form of the Index used by 
the mknufacturer and permitted a direct comparison of the 
difficulty indices obtained from the manufacturer's standardi- 
zation sample with those obtained from the language minority 
sample of this study. " 

Item discrimination was computed in terms .of point bi- 
serial correlations between each item and the rest of the <- 
items in the subtest with that item removed. , , • 

4 > ' * . ' 

Overall test ^reliability was computed in terms of Cron- 
bach's alpha. 



Development of the Testwiseness Materials 



Rationale 

Among the factors that are assumed to contribute to sys- 
tematic negative error, in the test scores of language minority 
students is a lack of testwiseness* tl^at is, the lack o£ spe- 
cific skills that could assist a student to receive a higher 
score than he might otherwise have received. The latfk of , 
these skills reflects a lack of experience with ,n6rm-ref erenced 
tests in the home country. 

In order 'to provide- the language minority student with 
those skills that he/she would have developed had he/she 
been born and raised in this country, a testwiseness train- 
ing system was developed and used in this study. 



Design arid Development 

The construct of testwiseness has a relatively short 
history in educational research. Both educational measure- 
ment specialists and test constructors have testified to its 



existence. It has been described, both as having a possible 
influence on reliability (Thorndike; 1951) and as a component 
of response .variance in objective test questions (Ebel and 
Damrin, 1960) . The literature reflected considerable con- 
fusion over the prefcise components and functioning of test- 
wiseness. No. empirical research to clarify the situation" 
was reported 'prior to 1964 . 

The classic theoretical work was performed by Millman, 
^Bishop, and Ebel (J.965) . They presented a comprehensive 
taxonomy of testwiseness that* was intended, to -serve as a 
framework for future empirical study. They defiried test- 
wiseness as "a subject's capacity to utilize the character- 
istics and formats of # the test and/or' test-taking situation 
to receive a high scofe." Subsequent to this work, a large . 
•amount of empirical ' research appeared in the professional 
literature. Within the last decide, testwiseness has been 
investigated in terms of its . coitiponents (Nilsson and Wedman, 
1974), correlates (Diamond and. Evans, 1972), and application 
jFord, 1973*) . A comprehensive review of investigations of " 
testwiseness has recently been conducted by Sarnacki (1979) . 
As a result of these and other analyses,, testwiseness^ is now 
widely recognized as a source of additional error variance 
in test scores and. as a possible depressor of test validity. 

Our first step in* the development of materials to teach 
testwiseness was to conduct an analysis of -the specif icf test- 
wiseness skills required by LEP students' in taking a stan- 
dardized achievement test, in this (Sase, the CTBS. This 
analysis involved three tasks: a review and identification 
of»relevant testwiseness skills contained ifi the taxonomy of 
testwiseness skills 'prepared by Millman et al. (1965),' a* task 
analysis of the skills required in the .'sub-tests of. the'CTBS, 
and discussion with experts in educational measurement and 
bilingual education about the problems faced by students 
generally in taking standardized achievement tests, and those 
faced *by students, in particular. ' 

Based on "the above analysis, a conceptual framework, for 
testwiseness training of LEI* students 'was developed (see 
Table 1)., Basically, three categories of skills were 'iden- ' 
tified as most important given the nature and purpose of the 
study: familiarity with standardized tests/ following .direc- 
tions 0 , and strategies for - test-taking/- ' . -\ 

• • * & 

c 

Our second step was to examine ^previously developed 
materials 4esigned to teach testwiseness skills to determine 
whether tjfcey would be appropriate for use in this gtudy. 
These materials were located through reports of studies pub- 
lished in professional journals, doctoral dissertations, and 
contacts with school districts across the nation reputed. to 
be training students in testwiseness skills. 

Our objective in reviewing. these materials, was to de- 
termine whether* they addressed the sfcills contained in our 
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TABLE 1 



CONCEPTUAL^ FRAMEWORK SOR ^TESTWISJINESS TRAINING 
OF LEP STUDENTS 



Familiarity with Standardized tes-ts 

* * »' * 

1. Features of standardized testa - « " 

2. Feelings when • taking-tests * 1 % ' 

3. Marking the answer sheet or test booklet quickly 
♦ and properly ^ , ' 

• • . - j ; .< - 

Following Directions 0 

v 

*• 1. Reading every word 

2. Understanding .types of questions and answers 



requirejd for each sub-test 

4 Reading Vocabulary 1 ^ 
Reading Comprehension 
Mathematics ^Computation 
Mathematics Concepts & 
Applications • ' 
Spelling . * 
Language Mechanics 
Language Expression 

^ Reference Skills 



CTBS. . 
Espanol 



% 

CTBS 
English 



.J 



Test-Taking Strategies 



0; 
V 



1. Strategies for answering feach tyj?e> of question 
in ^each subtest 7 

'2,' Guessing' . ' • 

3. Marking humbers of questions* for reconsideration 

4. " Using scratch paper . . « 

5. Checking answers 

\ - 



/ 
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conceptual framework , whether they were appropriate for LEP 
students at.. the grade ^levels identified for the stuciy (4th, 
6 th, 8th grade) , and 'whether they were usable* within the 
amount of 'instructional time available for the study 1 (5 one- 
hour sessions)-. A summary of the^key characteristics of all. 
materials located is presented in Table 2. As can be -seen 
from the table , the majority of materials addressed more 
sophisticated and complex skills and were directed vto older 
English speaking students*; Based. oh this review, it was' 
decided that no existing materials were appropriate, for use 
in this study. G , « < 

Thus, we began,, the process of developing testwiseness 
iriaterials^ for the study. A preliminary set. of materials was 
developed containing instructional 'strategies and exercises 
keyed to the CTBS* that. were appropriate for LEP student^ at 
•grades four, six, and eight. The materials presented 
struction on the ^kills identified in Table 1 in a- logical 
sequence with activities that were designed to motivate an4 
inform students. * 

" The materials were divided into five sessions. The or- 
ganizational format for -each session was, as follows:, set of * 
lessons, objectives, general ideas, activities, practice 
exercise, correct answers, and final assessment. The prelim 
inary materials were developed first in. English and* then 
translated into Spanish. * \ 



^ * ' , Tryout~ * * ./ - \ ' 

Upon completion of development of the material's; the 
student . materials were translated into Spanish and weVe pro- 
duced in small ^quantities. A tryout was. then scheduled with 
fourth, SLixth, 0 and eighth girade monolingual Spanish speakers 
The students came } from three different schools, one grade 
level, per school. Care was.takfen to make sure that they ^ 
would exhibit the same characteristics as s those exhibited by, 
the experimental sample. Class £izes ranged from eight to^ 
12 students. Four one-hour training sessions 'were scheduled 
kt .each 'site on .fouir. successive days. The classes were 
taught by a Spanish/English bilingual instructor and were, 
observed by ;the ^riter of the materials, a person who has 
good aural 'Spanish* ccSmprehehsion and is Spanish literate. 
The same material^ and procedures" were used with all three 
groups.* The only changes were in, the demeanor of the in- 
structor who tried in each ca?e to. talk with the students at 
their own ]$evel. Following * each one-hour session , the in- 
structor and observer .critiqued the session. The focus of 
these critique sessions was to gerferat^ a set^of specifica- 
tions for revisions ,df the materials*. m - 



TABLE 2 



HUSVIOUS STUDIES USING VfESTW IS IiWUHS ItlSTRU&TIOU 

raile 



. Skill (s) 

c 

Guessing 

Pacing of responses 



Graile 
Level 



ath 



Forma t 



Programmed Text 



Instructional 
Time 



70-90 minutes 



Using cues 



All Skills in Millman taxonomy 



Familiarise students with format ^ 
/(items with options, worR- 
^ ing in rows a,nd columns) 
Teach* common concepts (rigj|t, 
. loft, same, different) ' . 

Pay attention to all options j 
Practice marking 
Motivation -encourage to ask 

questions 
Use a marker • . , 

Increase attention span s 



college * 



9 th 



preschool 



IB (error 
avoidance) 



15 minutes 
30 minutes 

100 minutes 



1) script^ {prose) 

2) programmed 
text 

Outline of TW 
* principles 

Elaboration of 

c selected prin- 
ciples 

Teacherwiseness 
with individual 
study 



30 pages of prac- 2X a week for 6* 
tice materials . weelis 



Cue tiding 



J 4 



TABLE 2 (GontJ 



Study 



*Skill (s)< 



Level 



Format 



Instructional 
Time 



I 

H 



NT 



Slakter et al, 
(1970) ! 



< If 
Ca'llenbach 

4973) 



• Woodley 
(1975) 



15 



stem options 
absiird options 
similar options 
specific determiners 
guessing 

following .directions 
response marking 
using, time ' * 

guessing 

reducing test 
related anxiety 
♦ , 

time using ^ ■ 

error avoidance 

•x 

guessing 

Deductive reasoning 

Interest consider- 
ation 

• — # 

Cue using 



Seniors. C 



Programmed tests- 40 minutes each 



IB (error 
avoidance) 



2nd 



Adults 

(occupation 
& licensing 
exams) 



Instruction & 
Practice 



Workbook 



8 - 30 minute 
lessons 



4 hours 



16 



TABLE 2 (Cont*) 



Study 



Skill (s) 



Grade 
Level 



Format 



Instructional 
Time 



Ford & Wee£e£ 
(1980)' ' 



Test mechanics deductive 
reasoning ~ 

Cue using 



College 



Lecture 



Supervised study 
(3 booklets) 



3 sessions 



Johngma & 
Warshauer 
«<1975) 



Motivation 



Following , *direc tions , 
understanding what 
is read , 

Guessing 

Using answer sheets 
Using time 
Test-taking conduct 



5.th/inner. 
city £ 
suburban 



Teacher-led group 
discussion with 
written activi- 
ties 



1 hour 



1 hour, 15 minutes 
(lr2 sessions) 



Ferrell 
-(1977) 



Deductive reasoning 
Cue using ^ 



high school/ 
college 



Form 2 - a test 
as starting 
point for 
group dis- . 
cuss ion 



1 class period 
(1 hour) 



17 



18 



. • .Revision ' 

> 

The only revisions that were needed involved a rese- 
quencing of some of the content and activities. The revi- 
sions were completed. The testwiseness materials presented 
ttith this report are the revised version. 



Development of the Oral Administration Procedures 

a 

1 Rationale. m * 

* * 

I • ^ - 
Most TOung people in Latin" America do not have a chance 

at formal education. 'Nevertheless, since life in the home 
and community csfa be presumed to have educational value, we 
must -assume a .gr^t store' of practical knowledge. ^ Unfor- 
tunately^ due to a lack of literacy; these students can only 
reveal, this store through the use of oral ^language. 

If the purpose of testing is to find out what a student 
knows and if what a student knows cannot be determined using < 
a written 1 test, then it only makes sense to adjust the test-- m 
ing mode in order to accomplish- the stated purpose. The ob- 
jection may be raised that changing the mode of testing makes 
the results obtained non-comparable. This is certainly true 
in some, perhaps all, cases. An. oral version of a reading 
comprehension test, for example; cannot be considered a test 
of reading comprehension. However, it may still be possib^ o 
to learn something of value about a student's language and 
conceptual skills by presenting such a test in ,this way. 
And wheji it comes to certain other subjects (mathematics 
computation for example), what one learns with a test der 
ljjvered orally may be very similar tq> what' ojie lea'rns'from 
a printed, test. ' - 

These considerations lead % one to speculafce^about wfiat 
might be learned with regard to the state^of a student 1 '^ 
knowledge from an ofal, administration of the .CTBS. , Consider 
that the form *of the reading^ ^vocabulary items is as follows: 

large house 

(1) big 

* - (2) long 

(3) new"', ... 

(4) ready • • u 

In this item form, the student-is expected to choose the word 
that me&ns! the same or nearly the same -as the underlined word* 
In the printed version of the .test, the student must be able 



to decode the' words, assess their meanings, and make an appro- 
priate^ selection* ,ln the oral version of the test, only the 
stimulus situation changes. The student must still determine 
meanings and make an appropriate selection. "From such a' task, 
one can le^rn whether the toeaniijgs of the test words are 
knownjf 'The only thing one cannot determine is, whether, or the 
degree? to which, the written form of 'the words .is known.'' 



& similar argument can be made with respect to the read- 
ing comprehension subtest. Consider that the items present 
a shoprt reading selection followed tfy four-choice multiple- 
choice items. These items test for such things' as determini- 
ng tidji of word meanings from context, attention- to detail; 
sequehce of .events, mairr idea, etc. In the print version of 
the test, the student must first decode the reading selection 
and the items, then make the appropriate response. In the- 
oral version of tlje test, only the decoding part of 'the task 
is eliminated. The student must still deal with all 'the se- 
mantic requirements of the items. A similar argument can be, 
made for the mathematics 5 concepts and applications subtests. 



Pull recognition is given in this study to ' the fact that 
the oral approach almost certainly changes the measurement 
properties of the test. * £n assessment of these changes is 
included in the plan for data analysis * >: 

4 

The rationale, then/ for designing and developing an 
oral administration procedure rests on the assumption that 
students with limited literacy skills (whether English Or 
primary language) still have considerable knowledge which, 
because of the requirements of the print test, mu$t remain 
unknown to us if we limit ourselves to the use of print. 
Only as we adjust our testing procedures- to match the studen't' 
language capability can we unlock the door to his/her store of 
knowledge. 

Design and Development 

* 

In developing the oral administration procedures, the 
intent was to leave 'the form of the test unchanged wherever 
possible. Under .standard administration procedures, the 
examiner reads the directions. By slightly modifying the* . 
directions as giveri in the Examiner ' s Manual , it whs felt- 
that^the task of reading. the directions could fc>est be left 
witjh the examiner. The only^change would be to ignore the 
instruction 11 Do not read the item aloud" wHerever it oc- 
curred. This seemed th^f'most appropriate since the instruc- 
tion always appeared 'with Sample items and it was thought 
preferable* to present the sample items in a fashion identi- 
cal to the presentation of the items,, that is orally. For 
this reason, the instructions were not recorded on tape. - 
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TKe items for the reading. vocabulary, the reading com- 
prehension, ^nd the mathematics concepts and applications 
subtests were recorded'on tape. The mathematics computation 4 
subtest was not recorded since it does not involve written J 
language.^ This study did not deal^with the spelling, 'lan- 
guage mechanics, language expression, reference skills, 
science, and social studies* subtests since these subtests 
are not included in the CTBS Espanol. 

T^i preparing the recordings of the reading vocabulary, 
reading comprehension, and mathematics concepts' and applica- 
tions subtests, it was felt that pacing and understandability 1 
would be the; most critical • factors. * With regard to pacings, 
it was decided to record each subtest such that it would fit 
withip. the time limits specified for* the o pr j^nt version. 
Thus, the rian time for the reading vocabulary subtest had to 
-be less than, 15 minutes for .Level 1 c*nd 12 minutes for. Level 
3, .the ^reading comprehension subtest' less than 35^, for both 
Levels 1 and 3, the mathematics .concepts and applications 
subtest less than 35 minutes for both Lfev^ls 1 'and 3. « ' 

The actual run time vould of 'course be affected :> by the 
pade^of reading, the number of item repetitions , ^nd th* 
amount of silence' between items. It was- decided to read the 
items at* a pace slightly slower than normal, conversation*. 
The other decisions' varied amonc^ the test as follows:. ^ 

'1. 'On the reading j/ocabula^ry subtest, ~ v . v 

item stems were read once and re- . 
'sppnse choices twice. - Three -seconds 
of silence separate thq reading of 
each item. The total run time ojrf % 
the Spanish version, Level 1* is ^ * 
^12:50, leaving 2:10 for^those stur , 
'dents with reading skill to go back • 
over omitted or other items. The v 
total run time on the English ver- 
sion, Level 1, is 12: 08, ^leaving 
, "2:52. The total run time on the 

9£anish version>—£evel 3/ is 11: n h * \ 
leaving 0:16. Th6 total run time on \ 
the English " version, Level 3. i*s 
11:30, • leaving 0:30. \ * 

*2. On' the reading comprehension subtest, - • 
the H reading" 'selections are read^ ' 
* aloud once. Item stems* are read .once' \ 
and response choices twice. Three 
seconds of -silence separate the read- 
ing of each item. ' The total run .time 
on the Spanish version, Level l*is 
23:10, leaving 11 : 50 fpr . those stu- . 
: * dents' with some reading skills* to* go J % 

back over omitted or other items. 
The totai ruft time. on. the English " 
version, Level 1 is 22:45, leading' * 
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.12:15. 9 The tqtal run time on : the ^ 
Spanish version/ Lever 3 is° 26:00, 
leaving 9:00^ The toteal run time, on 
.the English, vers ion,, Level '3 is 24:11/ 
leaving- 9:49. - > 

pn the mathematics concepts arid.ap 1 * 
plications subtests, each item is 
. read once. Respoflse* options' are ( 
read' orfly 'where they enquire ' the 
X' * ' c reading of words . Twenty-three *se- 
k cond^of silence- separate the, head- 
ing of each item. The* 1 total run time 
on Jzhe , Spanish, ver siony Level is- * . - 
* 24:30 f leaving' 10:30 to gb back over 
oniitted or: cither it^jrts. \The total 
run' time on .the English version^ 
I^evel- 1 ijs 24:00'. Th4 tota^ run time 
on the Spanish versdon, Level 3. is 
- ' 24:15 f leaving 10:45. The total run 

time on the English* version , Level 3 / 
'■; is 24 :00 f * leaving 11:00. ' * 

. On .all the recordings, the 'beginning of each Subtest is- 
identified with' a tag giving t&e subtest plumber and title. 
Additionally/ each item is tagged by its A numbe;c in .the t6st 
book f thus assisting the student to -keep track of the item - 
he/she is currently working on'. * m \ ^ * 

With respect to the voices used on the recordings/ the • 
Spanish version was recorded , by a* male college prof essor\?f 
^Spanish whose primary Slanguage Is Spanish. The English ver- 
sion was recorded toy, a female 'prof essional narrator whose 
primary language is English; : 

The technical direction* and , editing was performed by a 
.'professional- musician" with years of studio experience both / 
as a performer and as a director/producer* 

<■ 

The recording wals done' in *a prof essional^ recording stu-„ 
• dio on Ampex equipment. / * ■ •! 



% 

Tryout and Re^i 
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An informal tryout was conducted in order to determine 
whether or not the recordings needed to -*be revised. Of -spe- 
cial cj&ncern was the effect of the* pacing f especially the 
length of the silences between items..* None of *the evidence 
indicated that changes were ^needed 'though it became clear , 
both in the tryout : sWt'ting and in the experimental setting 
that the time allowed is too long for- some' and too Ihort for 
others. The, fixed pacing is at cmce- the oral administration's 
greatest -advantage and # its greatest disadvantage; The advan- 
tage lies in the pressure /it places *ori* th6 examinee to respond 
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to every item. The disadvantage lies in .the degree to which 
it slows some student^ down "causing them to wait, and become 
bored, and speeds others beyond their normal pace, perhaps to 
their disadvantage. During tryout and ^ testing, no examinee 
wanted to use the available time at the end of each subtest 
to answer omitted items-. -As a consequence, the total testing 
time was reduced beneath the time limit s' recommenced by the^ 
test manufacturer. 



Selection o'f the Sample 



Problems Encountered 



*a The proposal on which ; this study is based was first sub- 

mitted more than three years ago. It j was rejected the first 
year. but along with the rejection came, some suggestions for 
w modification and an invitation to submit the modified pro- 
* ,,posal. The modification^ w^re made and the invitation s 
accepted. The time lapse between* the original submission and 
funding of the re-submission was approximately two years . An- 
— other six months passed as the start-up work was completed. It 
was now time to begin' preliminary work with the cooperating 
school 'districts. It was at this time rthat two of the three 
original districts decided that they could not participate. 
One of them had decided just* that year to discontinue use of 
\^ the CTBS, a key instrument in this study, and to begin using 
.the California -Achievement Test (CAT). -We now faced the 
problem of seeking £t a late hour additional districts that 
would cooperatq with us„ t Such districts were found, but ^ 
onl^at the expense of a reduction in "the sample size ori- 
ginally planned. Later, because of internal communication' 
problems, one of the* junior ' high principals whose school was 
scheduled to supply 30 eighth grade subjects, declined; to 
participate^ * 
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Selection 'Of Participating Districts 

As noted- above, one of the three original districts Con- 
tinued in the project. This district (we shall designate it 
as District A) l*ies on the eastern side of a large metropoli- 
tan area adjacent to an area known as a "port of etftry" for ♦ 
Mexican immigrants. This district has a large population of 
Spartish-surnamed students, a large and growing population of 
Asian students, and a large population of "Anglo" students. 

r 

' Of the new districts, District^B lies on the southwestern 
side of tka^metropolitan area and has a predominantly Black 
population that is being increasingly displaced on its' west 
side by inun£gra]^;,Latin American students , primarily Mexican - 
and Cfentral American. 

. . • • .V- 
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District C is on the northern fringes* of the large metro 
politan area. It is. t6tally different in character from Dis- 
tricts A and B in that it is predominantly middle ^to upper- 
middle class "Anglo" with a- strong sense of pride in its 
traditional support o£ education" and the arts. There is a 
growing immigrant Hispanic population ^iiviog on the outer 
fringes of the -city. 

■» - 

Districts D and E are in adjacent suburban cities lying 
in what may be called-an "ethnic corridor" to the southeast 
of the large metropolitan area. District D has a large popu- 
lation *o£ Mexican Ame'r icansxand "Anglos." District E is 

■ethnically diverse. District D participated fully in the 

study while District E agreed only to supply, non-experimental 

control subjects. * " 

• " ♦ 

/ ■ 

« 

Selection and Matching of the Students 

o The student participants were selected in a way designed 
to " facilitate the formation of groups that would be as nearly 
a?like as possible. Since we had a limited set of available 
'.students, random selection from a population was nor possible 
We therefore decided to - approximate a matching procedure . 
Approximation was a necessity since strict matching* is waste- 
ful of subjects and we were in no position to give up more ♦ 
thkn'3 very,* few of the available sample. • 

The actual procedure required that we define a set of 
variables on which the approximations would be based. The 
variables and their associated values were as, follows: 

1.9 grade in school (we used fourth, 
sixth, andv. eighth grade students) 

2. country of birth "(students were clas- * 
sif ied 'in terms of whether they had 
been born* in the United States or in 
Latin America)^ 

3. * amount of formal education (students 

'were classified in 'terms of the ac- 
* tual number of years of formal educa- * 

tion they had received* both in the % ; 

United States and in- Latin America) 

4. father* s level of education (students 
' * were classified in terms of the ac- 
tual number of years of f ormal .eduda- 

tion their fathers had received) 

• ^ 

5. - time in the United States (students 

were classified in terms ,of the ac- 
tual number of years "they had lived 
in the United States) 
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6* English proficiency (Each student was 
placed in one o^ seven categories 
based jointly on scorgs from the tests 1 
used in the participating districts 
for purposes of "program classification 
and teacher judgment. Tests Varied 
among; the Language Assessment .Battery , - 
the Bilingual Syijt&x' Measure , and the 
• , . Language Assessment Scales.) 

7. Spanish language proficiency (Each 
^ student was placed in one of seven 

• categories ^Based jointly on tests for 
language- dominance and teacher judg- 
ment . ) " ; s 

Data for each of 'these.* variables were coded on cards like 
the facsimile presented in Exhibit 2. Once the cards .were 
filled out, they were Mparated by grade level i Then, working 
one grade level at a time, the cards were placed two at a .time 
in two stacks corresponding to the two training groups. Each 
pair of cards w^s matched on the s*even variables as closely 
as possible. The bilingual resource teachers assisted in this 
so that approximate equality between 'groups would be obtained. 
'In all cases, these teachers were satisfied that there was no 
systematic bia§ favoring one group or the other. 

Following this, step, cards in each of ^ the two piles were 
distributed over four piles corresponding to the four testing 
conditions. -This was -done by (1) identifying sets of four 
students who were as nearly alike as possible and (2) assign- 
ing them randomly to> testing conditions. Oijce again, the re- 
source 'teachers were satisfied, that the groups were as neatly 
equal as they could be made based on the available informa- 
tion. Theivnumbers of students assigned to each condition is 
presented in Exhibit 3» Note that there, were no fourth grade 
students in fche n no- testwiseness/oral administration" groups. 

Assignment of Students to Conditions 

Once'' students, had. been assigned to groups such, th^fc^the 
groups were as nearly equal as°possible, the groups were 
assigned randomly to treatments. A totjal of 88 .students was 
in the group* receiving Level 1 of the test'. A total tof 73 
students was in the group receiving Level .3. . * 

Training in Testwisehess 

Students' who had been selected to receive training in 
testwiseness received their training no more than two weeks 
prior to taking the CTBS. The training was conducted jointly 
by the investigators and by personnel from" the participating 
districts;. In this section, the training procedures, train-, 
ing schedules, training environments and other training fac- * 
tors are described. * , 



Name 



School * 



• EXHIBIT 2 
STUDENT DATA CARD 

■ - * "District^ 

4 Grade 



Country of birth 



' Years oF- father's 
# education 



English proficiency_ 

* ♦ 
CTBS English %ile 

Readi ng • 



Years of education 

Time in U.S» 1 ' 

Spanish proficiency 

CTBS Espanol %ile 
Readi ng 



Math 



Math ♦ 



\ 



Assignment to Treatment 



Language of the test 

English 

Spanish 

Testwiseness training 

Yes 

No 

Admin4stration ' » 

Standard 

Oral 
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EXHIBIT 3 



DISTRIBUTION OF , PARTICIPANTS QVER CONDITIONS 



CTBS Level 1, Grade 4 



f* 
1 


• Testwiseness / 
.Training- 
n = 66. 


. — — 

No Testwiseness 
\ Training 
> n = 22 • 


Oral 
Administration , 


Standard 
Administration ' 


Oral 
Administration 


Standard 
Administration 


Spanish 
version 


n; = 20 

* 


n = 20 


n = 0 


n = 11 


English 
version 


a = 13 


n = 13 


n = 0 


n = 11 • 


9 


, ^ f 

/ 

CTBS Level 3, Grades 6 and 8 

\ 




j » 

^ Testwisenfess 
Training 
n = 4L 


Testwiseness 
Training 

n = 32 * 

- v 


' Oral 
Admini stra tion 


Standard" 
Admini s tr a tion 


Oral 
Administration 


Standard 
Administration 


Spanish 
Version 


n - 11 


. n = 8 . 
* 


n « 6 


* n = 11 


English 
version 


n - 11 4 ' 


n - 11 


A n m 4 

/ 


< 

ri = 11 * 
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Training the Trainers - * t 

""Three persons in additiqn to the principal investigator 
were scheduled to* conduct- the training-. The three" persons' , 
each came from a different district, one from District A/ * . 
one from District C> and one- from District D. .The principal 
investigator "conducted the training in District B. No stu- 
dents from District E received training in testwiseness. 

Since the principal investigator had been (1) the trans- 
lator of the testwiseness materials, (2) v .the tryo'ut, teacher , 
".and (3) the one who actually wrote the revisions, it wag 
natural that hp be the trainer of the three persons who would * 
conduct testwiseness training. - 

Th* three trainers plus other representatives of dis-^ 
tricts A,B,C, and D came together for an all-day training' 
session. During the first two hours, the purposes and proce- 
dures of the study were explained in detail. The next three 
hours were dedicated to going over the materials and proce- 
dures, for conducting testwiseness training, "The majority of 
the participants vjere Spanish/English bilinguals. AccSrdingly, 
their advice was sought regarding the accuracy of the trans- 
lations. An attempt was> made to rid the materials* o£ voca- 
-\bulary that wou^d be strictly regional. Their suggestions 
were incorporated into the final .revision of tiite Spanish 
version of the materials. The final hour of the day was 
dedicated to instructing the trainers jfn procedures for se- 
lecting participants and for assigning them to treatments. 

By the end of the day, -all of the persons attending the 
training day expressed enthusiasm for the project . \ Each 
' f e^t well-prepared to go ahead with his or her part' of the' 
study; ' v , * 

At the request of representatives from Districts A, B, 
• and C, the, principal investigator visited the respective 
districts tQ provide additional "assistarice in sample 'selec- 
tion. In District E', the principal: investigator wa'b solely 1 
responsible for sample selection. 

< 

Training Schedules 
* * * 

All of the participating districts wished to avoid bur- 
dening the students with testing over and above that which is 
normally 'conducted at the end of the school year. As a re- 
sult., training was conducted as early as mid-May in one 
district and as late as mid-Uune in another. The unique 
constraints in each district forced us to accept a variety 
of testing schedules. * In District A and D, four one-hour 
session^ were conducted at each of two schools. In District 
B, .two two-hour ^sessions were conducted at one school. In ' 
District C, one fdur-hotir session wafs - conducted at each of 
;two schools. In all cases at least four hours of instruction 
r were given. ^ * ■* 
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> Conducting the Testing 

The testing, like the training, was the joint responsi- 
bility of the investigators and school personnel. Testing 
in Districts A and C was conducted by .the game persons who 

• conducted tesfctfiseness training. Testing in District E was 
. conducted by the District's classrpom teachers according to 

standard District procedures. Testing in district B was con-? 
ducted by the investigators only and iru District D by the 
investigators and the testwiseness trainer jointly. 

0 

t 

% Tn District B, two examiners were used. Students were 
pulled out of their, classrooms in small groups and were 
taken to special rooms where distractions would be at a 'min- 
imum. Each group was tested on two successive days. Groups 
that .were tested in the morning returned to their classrooms 
k-Ain ihe afternoon and vice versa for those who were tested in 
the afternoon/ Students were seated at separate desks so as 
to minimize the likelihood of copying. In the> case of the 
students who received the oral administration condition, the 
tape recording was played on a single tape . player with the . 
volume turned up to a^cdrafortable level. As far as could 
be determined each student could hear the recording 
All sessions were carefully proctored. 

* 9 * 

In District C°, the stjidents were divided into four groups 
according to testing conditions and taken to a trailer where 
they ftould" be protected from distraction. Each group re- 
ceived the whole test (i.e., four .subtests) in one day. 'The 
four groups received the test, on four successive days. The 
sessions were carefully proctored. ^ 

In District D a , four examiners were used. Students were A 
pulled out ^of their classrooms in small groups and were taken 

* to special "rooms,. The students receiving the 'oral, adminis- 
tration procedures were taken to a medda center consisting 

"of two adjacent rooms where listening posts were, 0 available . 
The „tape recording was played to them through headsets with 
individually adjustable volume controls. "The use* of the 
listening posts' dictated that students be seated at the same 

'table. Accordingly > proctoring was tight in order to mini- 
mize the lifcelihodd of copying. In the case of tijhe students 
who received the standard administration condition/ seats 
were^ sufficiently far* apart to eliminate the possibility o'f 
copying. ..These sessions were also* proctored carefully. 

^ In District E, students were tested in their classrooms - 
6y their own teachers according to standard district proce- 
dure . ' 

The total humber "of^students taking 'the CTBS/S was 74,/ 
The total number taking the CTBS Espanol was 87. • 



Analyzing the Results . 



t . Changes in, the Design 

&t the orientatiqn and training session in which the 
testwiseness. trainers were trained , objections to the pretest/ 
posttest feature of the desigh were raised by the School dis- 
trict-representatives. They felt that .the .pretest should be 
avoidaajLn . order w to_avbid over-loading the students with 
testing. Classroom "teachers tend to feel that testing is 
overdone anyway* The representatives therefore felt that .the 
teachers and students both'^would be : more receptive to partici- 
pation if we could' go to* a posttest-oi^ly design, « The invesi- * 
gaLtors had no objections t$ this change since it was felt that 
adequate controls. fcad been' built * into the sailing plan, thus 
increasing the likelihood that the results * would be interpret^ 
able in terms of the variables that wer6 under study, - The 
Program Officer was t consulted regarding. the change to a 
posttest-only design • She had no objections,' : Accordingly^ 
the study was conducted in two, stages. During 'the first 
stage, testwiseness training was given to 'the students who 
had been selected for su6h 'training, . During the second stage 
the studerits were tested using one of four conditions as * 
follows : 1 : * ' • 



1;. CTBS in English/standard ^administra- 
tion, procedure * 

2. CTBS "in Spanish/ standard administra- 
tion -procedure 

3. CTBS in;Englishyoj:al administration 
procedure o 

4. CTBS fin Spattish/oral administration 
procedure - 



^ . . • 4 *?he Analytic Approach . 

As described xn a previous section of . this report , the m 
study was designed *as .an investigation of three experimental 
f^ctorst receipt of testwiseness training, form of test ad- 
ministration ; and language of test. £ach of these .factors 
wajs- studied at two 'levels: testwiseness training vs. no 
testwiseness training*, oral vs. standard test administration, 
and English vs.. Spanish Version of the CTBS*e The vlevels 'bf 
these factors were xtonsidered as fixed and the -resulting de- •* 
sign was* a 2x2x2 factorial with students serving as repli- 
cates within each of the * eight treatment cells. 
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Analysis of tlfis design produces a maximum of seven 
terms corresponding to the sources op variance , in ari analy- 
sis of variance table. 1 These terms/r.epresent the three main 



\ 



3tf 

J.J 



\ * 



effects (testwiseness training, fSrm^ of * test administration , 
afid language of test) , the -three- two. way^ interactions (test- 
wiseness and administration) , and finally, the three-way * • 
interaction, term (language and testwiseness and administra- 
tion) . Estimation and significance testing of each of these 
terms is possible as long as there are replications within ' 
the design cells so that the residual variance can-be sepa- 
rately estimated. 

At Level 3 (grades 6 and 8) , each of the eight experi^ 
mental cells had more than two observations, thus allowing 
estimation $nd testing of the complete model (all seven vari- • 
ance sources). However, at Level 1, two of th e cells werfe - 
empty, producing a reduced model capajaJ^Q^ estimating and 
testing only five terms.. Since botKempty cells involved 
the condition of oral administration and no testwiseness 
training, the two-way interaction terms between these two 
sources and the three-way interaction term could not be se- 
parately estimated. Therefore, the residual variance term 
for this model included these sources of variance. Por the 
discussion of this design, the reader may consult a standard 
experimental textbook such as Winer (1962) or Kirk '(1968). 

Actual computation of the estimates and their associated 
significance.- tests was accomplished through' the use of the 
Statistical*, Analysis System (SAS) procedure labeled TLM. 
This procedure was chosen because both the Level \ and Level 
3 designs were' unbalanced (unequal cell frequencies) and this 

'procedure would compete the correct sum of squares for such 
designs. The -procedure ^ses a general "linear model approach 
to allow the specification and estimation of a w:j.de range of 
analytic models. While this procedure estimates four differ- 
ent types of estim&ble functions, the relevant* type for the 
current analysis, is the Type IV function." This function 
allows for the testing of ^the hypotheses as if the data had. 
been originally derived from a balanced design. More spec;L- - 
fically, the Type, IV function estimates sum of squares as- 
sociated with adding each term as the last term to the linear 

"model. In other words, it computes the sum of squares for 
each term holding all the other terms of the model constant 
(see Helwig and Council , 1979). - 

All computing was performed at the University of Southern 
California computer facility using an IBM 370/58. 

The' results of these analyses are reported and "discussed 
in. the t following section. *. ± 

The s*)cio-linguistic item analysis referred to earlier 
was structurally unrelated to the 2 i£^2 x 2 factorial analysis • 
described* above * - In other words,, there was no attempt to; 
study the effects of the Varipus treatment conditions on indi- 
vidual item response tendencies. The socio-linguistic analysis 
was designed to staxjd alone ancl to. provide a getter understand- 
ing of how Spanish-spe'aking LEP 1 students as a group respond- to 
paired English/Spanish items . • ' 
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The Linguistic Analysis * 

As was mentioned earlier two approaches to item analysis 
were used* A linguist examined each of the items from a socio 
linguistic standpoint. This analysis* was conducted on Levels 
B f C, l f Zf and 3 of both. English and Spanish versions of the" 
tests* Additionally, a statistical analysis was limited to 
Levels 1 and 3 since / these were the only levels actually ad- 
ministered to subjects in the study. • 

> *" - . • 

In v presenting the results of the analysis we shall first 
deal with a comparative analysis of the overall distribution 
of item difficulties across *f5ur different sub-populations. 4 
Following this, we shall discuss the socio-linguistic and 
statistical analysis" in an integrated. fashion. 

The comparative analysis is based on item difficulties 
expressed' as the proportion of correct responses among four 
distinct grqups. The groups are (1) the norming sample used 
by the v manufacturer, for the CTBS/S, (2) the sample of LES 
students from this study who.^ok the CTBS/S, (3) the norming 
sample, used by the manufacturer for the CTBS Espanol, and 
(4) the sample of LES' students from this study who took the 
CTBS Espanol. The relevant data are found in Tables 3-10. 

Consider first the relative performance of the English 
norm sample (Column 1)" and the Spanish study sample (Column 4) 
This comparison is of interest because of the fact that it 
involves the two' primary" groups for whom the , two different 
language versions were prepared. Other things being equal/ 
one might hypothesize that item difficulty levels would'" be 
approximately equal* f or each English/Spanish itenrpair. 
Examination 6f the tables reveals that this hypothesis holds 
for a significant number of the items. If' we define "approx- 
imate equality" as item pairs whose associated difficulty 
indices are within. + .10 of one another, we -find 19 out of 
the 40 items on.*he~Level 1 Vocabulary subtest that can be 
said to be of approximately equal difficulty. On the Level 
1 Beading Comprehension 'subtest, 32 out of 45 items' were of 
approximately equal difficulty. On the Level 1 Mathematics 
Computation stibtest, 41' out of 48 items were of approximately 
equal difficulty. On the Level 1 Mathematics Concepts and . 
Applications .subtest, 22' out of 50 items, were of approximately 
equal difficulty. Similar ^proportions of equally-difficult 
items* Were found ori the Level 3 subtests. Of course, the ". ' 
remaining English/Spanish item" pairs differed by more than + . 
.10; The differences' were occasionally as-great as + ."60. 
In all but a few cases, differences larger than. 4 .10 favored 
titie "English' language sample. In other ' words,.. if *"an item pair • 
^exceeded the + ^.10 difference in difficulty, the item was 
easier for the' English language sample than for the Spanish 
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TABLE 3 : Item Difficulty in Proportion of Correct 
Response for Four Reference Groups: 
Vocabulary Subtest ' 



/ 

r 




Level 1 


CTBS Espanol, Level 1 




X uCUl 


Norm Sample 


\ ■ 

Study Sample 


> 

Norm Samp la 


Study Sample 






.94 


.71 


.89 


• 

.95 • 




2 


.88 : 


.36 


.78 


.97* 




3 


.90 


.57 


.52 


.90 




4' 


.86 


.50 


.39 


.46 




5 


.85 


.60 


- .47 


.62 * 


> 


6 


.86 


.50 


.49 


.78 




7 - 


.77 


. .22 


.38 


.26 




8 


. .80 


.50 


.39 


.56 




9 


..80- 


• .67 


.64 


.82 




10 


.84 • 


« .63 


.39 - 


.54. 




11 •' 


.78 


.48 


.24 


.16 . 




12 


.71 


.44 


.47 "\ 


.71 




' 13 


.64 


.39 


% .56 


.79 




*' 14 


.80. 


.50 S> 


.38 


. .55 




15 


• .62 


.61 


.46 


«. .87 . ' 




16 


.50 * 


.54 


.85 * 




17 


.75 


.80 


.74 


.85 




-18 


.70 


•59 


•55. 


.62 




• 19 


..67 


.42 


.66* 


.74 


• 


20 


.63 


•32 


.51 


•72 




21 


.77' 


.39 " 


. 53 


.87 




22 


.68 


• 17 


.46 


.64 




23 „ 


.62 . . 


.29 ' 


.31 


,18 




24 


.70'. 


, .54 


".54 


.77 


* 


25 


.61 


.26 


.35 


.77 • 




. 26 


.63 


.43 


.46 


•76 




27 ' 


.56 


.32. 


.36 


.66 




28 


-.53 


.18 


.51" 


" .66 




• 29 


.51 


.48 


. .26 


.32 


l 


'30 


.51 . .• 


* .05 


.25 


.18 " 




31 


.58 


• »• -50 


.39 


„.82 




; 32 •. 


' .-74* • 


.33 


, • .35 • 


. '.44 




JJ 






.26 


.13 




'34 • 


.45 


* .33 


.38 t 


. .5\> 


r. 


35 . 


'.52 


.00 


' .43 „ 


.45 




36 


.'49 


.36 


.49 


.76 


• 


37* 


.53 , 


.19; 


.2.3 


.49 . 




" . . 38'.' o , ; 


•54 


,27 


. , .26 


".14 




39 ' 


• 50. ' 


.24 


.34 


.42 




40 




.38 


.29 - 


.54 




* 






* 






♦ 0 


t 


* 









mum 



. 3 
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TABLE 4 : Item Difficulty in Proportion of Correct 
Response for four Reference Groups: 
* Reading Comprehension Subtest 



Item 

1 

2 

3 

4 

5 

6 

7 

8 

9 
10 
11 
' 12 
13 
14 
15 
16 
,'17 
18- 
19 
20 
21 
22 
« 23 t 
'—24 
25 
26 
27 
28. 
29 
30 
31 
32 
33 
34 
35 " 
-36 
37 ' 
38 
39 
40 
^41 
42 

43 , 
44 
45. 



JERJC, . \ 



CTBS/S,, Level 1 


CTBS Espanol, Level 1 


Norm Sample 


Study Sample 


Norm Sample 


Study Sample 


« 

.71 




.56 


59 


/ 76 


.86 


r 


.52 ( 


fi7 


79 


•66 




^32 


.45 


.76 


79 




44 

' < 


55 

. J J 


79 






64 


64 

^ • o*» 


. lV4tm 


.82 




72 


58 

. JO 


73 


.87 




, . < .58 . 


58 


73 if 


73 




79 

• * 


59 

• 33 


62 f 


.53 




18 


44 
. **** 


38 . 

. JO ' 


78 * ' 
• * o 




33 


46 
• *»o 


• / j 


40 

• 




■ ' 13 


42 


79 


% * 67 




46 


38 


47 * ' 


61 




• 35 


40 


■ « . fi7 


76 




29 


41 


45 


63 




17 


31 


38 

. JO 


* .82* 




' » 65 


51 


68 


79 




^25 ' 


53 
. j j 


68 f 


74 




.18 % 


.49 

1 i 


.43 


• 65 




.26 


39 1 


35 
. j j 


* . "SO 




13 

* 


32 


29 


76 . 
, • /o 




.36 


'43 


63 

• UJ 


68 




4 42 


37 


51 
• ji 


66 




.38 


47 


69 


. o j 




42 


41 


64 


61 

< . or 




21 

Jt) ****** 


37 


56 


.75- 




40 


48 * 

* • "o 


79 


• 73 * 




56 

. JO 


40 




* ' 78 
• / o 




.52 . 


4 .52 . % 


/ .80 


.76 




35 


48 


1 J 71- 


.67 * 


• 


29 


• 44 


- • fi9 


•*78 




48 ' 


47 


. oo 


.62 , . 




.28 


.30 


.45 


' - , .64 ' • 




.50 


•'44 


.63 


* .68 




.67 


.36 


.66 . 


.79 




.38 


.5*1 ** 


».71 


".75' • 




.57' 


.42 . 


.49 


' .71 . 




.71 


.49 


,74 


.68 




.39" 


.45 


.60 


.47 




.29 • 


.26 


.24 


.52 . 




' .17 


.34 


' .54 

.T3T , 
. .46 - 


.77 
.60," 




' .74 


• .48 




' „ .32 


- .38 - - ' 


^ * .63 




.33 " 


.3*2 ? 


.51 


&? ' - 60 

W .52 ' 




.48 


.23 


•,34* , 




. . .39 


.36 - 


.63. , 



SLA, 



^ *TABLE 5 : Item Difficulty in proportion bf Correct 
*, t Response for Four Reference Groups: , 

* ? r Mathematics Confutation Subtest 



Item 

1 

2 



5 
6 
7 
8 

9 
10 
11 
12 
13. 

14 , . 

,15 \' * 
16- 
17 
18 

• 19 * *. • 
20 • 
21 
22 

23 % • 
24 
25 
26 
27 . 
28 
29 
30 

31 ' 
"32 
33 

34 ■ 
35' * 
36 

37 m 

38 

39- 

'41 

•* : 42 
43 

44 • * - 
• 45 

, 4&, 
"O' 47 

RIC 49- 



CTB5/SV Level 1 



CTBS 'Espanol, Level'. 1 



Norm Sample 
• 


Study Sample 


* Norm Sample* 


~ — ~ ' 

Study Sample 


.94 • » 


1.00 v 


— 73 

Hp 

\^95* 


1 JD0 




.96 


.97 


1.00 , 


.91 * 


»*, ...92 


.93 


i,po 


f • .£8 


.84 


.91 ' > 


, .87 


.85 


.88 


.87 


.92 




* .75 


.85- 


.92 


.81 


.76 . 


.85 


• 87 


.89 


.88 


.90 s 


.97 


.84 


.75 


* .87 - 


.92 


.77 


.84 




.78 


• 82 


.68 


.83 . 


.92 


.73 




.72 


*\75 


s .88 


.75 ' 


.90 


.97 


• 80 


.56 




..84 * 


.77 . 


.68 


.75 


.89 


.84 


.92 


.86 ^ 


-.97 


.81 - 


.88 


.85 


.89 


.78 


.76 


.82 


.B7 

* > 


- .79 


v * .72 


.78. < 


.89 


.70 


• * ."45 


.69 




> fc , .83 


.96 - 


* .82 


. ..95 


.80 
* 


.83 


.78 


' •«£ 


.68 


.48 


.70 


>£ 8 < • 


• ' .65 


• 52 


' - .63-' 


^64 


.89 


1.00 


K 92 


.95 , . 


.92 ~ 


* ,1.00 > 


.92" 


1.00 


.87 


? 92 


. . 91 


.92 


.88 


rj .88 


.88 


.95 . 


.-81 


,84. 


.81 


1.- . 
. .69- * 


.73 


.46 


<77 


r • 


' .75 


-.63 


.80' 


.78 


.82 


.96 . 


.85 


.8? 


.76 


.58 


.79 


_ .70' 


.84 


-.92 


.86^ 


.92 


w .73 s 


.71 


.75 


.76 


' .69 . 1 


v -63 


.75 


.76 ' 


.85 


••92 


.87 


.89 . 


.86 \ 


.83 


. ♦ .87 - • * 


' .95 


: .70 


.76 * 


* ; .76 


- .84 


■ .87 


-.92 ' 


.87 , 


.92 


/.83 


.88 


.87 


.86 V , ■ 




.88 


.86 


.86* • 




.88 


.86 • 


' .86 


' ' ~-.82 ■ " 


.78 


• .84 


' .8*4- • 


.80 • 


' . • .83 


.81 


.84. 


.63- 


.38 


.70,- j 


' -38 


• .65- 


%- -26. ' • 


.'67 


.43 , . 


.63* 




.69. 


.55 . ' . 



TUBLE 6 : Item Difficulty in ^Proportion of Correct 
Response for Eour Reference Groups:^ 
Mathematics Concepts* and Applications Subtest 



„ w 



Item 

y 

1 

2 
3 
4 
5 
6 
7 
- 8 

. 9'' 

la 
n 

12 v 
' i3 , 
14 
15 
16 
17.< 
18 
19. 

- 20 
-21 

. 22 

- 23* ' 
24 

. 26 ' 
27 
*28 
29 
30 
31 
32 
33 
34 - 
.35 
36 
37 
. 38 
39 
40 
41 
42 
43 
'44 
45 
.46 
-47 
48 
49 

50 • 



' * • • CTBS/S, Level 

* * 


1 


CTBS Espanol, Level 1 , 


No£m Sample 


" - ! 

^£udy Sample , 


, Norm Sample 


Study Sample 


*> 


* * 








-« 


, * .55 


• .0,' 

< * 


.29 . * 


> 


* .28 


.24 


■ - .85 ; 


< 


.84 




.77 


.89 


.90 •• 




.88 




• 81. 


^.89 


.79. 




.76 




. 71 ( 


.84 


.90 




.88 




84 


97 


.85 




.64 




v ! .72 


.87 


.88 




.60 




' .73 


78 


.71 




.56? • 




.58 


.79 


.90 » 




. .88, 




.79* 


.92 


.83 




.80 




.75 


.76 


' .84* 




.88 




.79 


.95 


.82 




.63 




.$2 * 


.sr 


4 .74 

> 




.45 * « 




.59 

<• 


* .53 


■ .51 




.39 




.48 


* .*39 " 


' -£3 . 


» . 


.43 ; 




*49 


.29/ 


.76 




.48 v 




.58 


.71 

• 


.78 




.48 




" .66 


.73 


- ^ .89 ' 


J. 


:92 * 1 




.78 


.97 


.74 


.35 ' I 




.62 


.50 


.86 


.79 / 




.74 * - 


.92 


. ♦ 1 .-70 


* 


.44 




0 .57 


.82 


.66 




.36 




.54 


.56 


.66 . 




,. 4 17 




.48 


46 


* . .60 


• 


.14 




.35 * 


.43 


.67 


* 


.50 




.60 


63 


.81 




.76 * 




.68 


• 7 -* 6 : 


.80 




.54 * 






.82 


.79 




.54 




.55 


45 

• 


.67 




.17 




**f54 




.72 




.57 v . 


M 


, .55 


.53 


.85 




.59 




.59 


.61 - 


.-75 




.30 




.52. 


.62 


a .55 




.33 




.37 


.35 * 


.56 




.15 % 




.38 . * 


.35 


.79' 


1 






.63 


.57 


. - .55 




.13 




*-.S7 , '." 


.35 


.79 




.50, 




.70 r ° 


k .63- 
.47 


.72* 




.43 




.52 


.71 . 




.18 . 




• 56 


"47 , . • ' 


, .60 




• .47 


a 


' .47 


65 


.78 • ~* 




39 ' 




.60 


.70 


.80 " , 




.48 .- 




. 


.58 


.69 




.59 j 




.47 . 


.43 






.46 - ' 




4 .54 


.50 - 


• .70 




.35 




• y.59 


.58 •• 


.65 




.17 




.46 . 


.45 


.70 • - 




.65 






.63 


'•" .63 : V. 




.22 




. ' -43 


.32' 






:43 




.60 • ; ■ 


" ' .76 


« . .66, ^ •. ,. 




-30.-; 




•57 


• ' 41 • ' 



TABLE 7 



Item Difficulty in Proportion of Correct 
Response for Four Reference Groups: 
Vocabulary Subtest 



- 

CTBS/S, 


Level 3 


CTBS Espanol, Level 3 


No Inn .Sample 


Stgidy Sample 


Norm Sample 


"study Sample 


* * 

.88 


.43 






.82 


. .67 


.77 


.43 




• 


.62 


..46 • ' 


.80 


.43 






.70 
. 2 .71 


.38 


.76 


.39 






- .58 


.85 


„ .52 


o 




..78 


.79 


* ,79 


.39' 

V 






.81 


1.00 " 


.83 


.22 






.49 


.75 


.77 


SO 






. 39 


• .63 


.79 


.61 






• 71 


.92 


.•65 


.22 






• .47 » 


.50 


.83 


.78, 






. .67 


.75 *• ' 


.69 


.39 






.73 


.=88 


•72 


.35 


- 




.57 


.75 




-.61 






.55 


.71 • 


.77 


.70 






.73 


.75 


.71 . 


.65 






.87 - 


. .92 . 


.61 


.22 






-.58 


;96 




.43 






.56 


.71 


r *\57 


.17 






.73 ; 


• .75 


.6-2 


.35 


4 




.37 


.67 


.67 


.30 






.44 


.63 


.74 


.35 






.63 


.86 


y.65 '* 


.43 






.44 


.21 


.-61 


.35 






.43 


.58 


.62 


.32 






.61 


.00 . 


.63 


• .41 " 






.52 


.'54 


.62 


'.59 






.-54 


.71 


■ » .61 


. .18 






.22 


.25 


• .50 


.33 






.51 


■ .63 -. 


\65 ■ 


-.29 






.39 


.21 n 


.'55' 








.67 . 


• .75 


.*S8 - 


.38 






.52 


•67 


.56 


' .43 






.47 


.54 . . . 


.53 * 


.57 






-.54 


.48 


.48 


• - .38 






* .56 


.59 , ' 


- '. .49 ' 


.45 






.37 


.64 - 


.48 


.42 






;25 ' 


.36 


.46" 0 


.21 






.23 


-.36 


.50 


.47 






-.39 


.68 


.49 


" ,21 




y 


.48 


.87 

* * 








• 






' 031 

-31-...' 


37 




t 





TABLE 8 : item Difficulty in Proportion of Correct 
J Response for Pour Reference Qroups: 

Reading Comprehension Subtest 





CTBS/S, 


Level 3 


» CTBS Espandl, 


Level 3 


> 

• Item 


Norm Sample 


Study Sample 


Norm Sample 


* ' 

Study Sample 


1 


.90 


.70 ^ 


.78 




.88 


2 


.78 


- .35 


.81 




.88 


3 


' .71 - 


.22 


.59 




.75 


4 


,69 


«• . .39 


.60 




.63 


« 5 


- .67 


- ' '.39 


.47 






. 6 


.72 


♦ ' . .59. 


.56 




\ .43 


. 7 


* .75 . 


.23 


.57 




.50 


8 


.58 


' .43 


.47 




.75 


9 


.53 


.13 


/ .34 




.50 


10 


.49 


.17 


.44 




.29 


ii 


.81 


.83 


.68 




.92 


12 


.63 


.48 


.53 




^ .65" 


13 


■*' .68 


.13 


.46 y 




.63 


14 , 


• .55 


.39 


.36 




.50 


15 ' 


.38 


.22 


.30 




.37 


16 


.49 


.48 


.26 




.30 


17 


.80 ■ 


.39 


.59 




-.63 


18 


.82 


.70 


• 65 




.80 


19 


.72 


:30 






.60 


20 


.68 


.35 


.53 




.77 


21 


? :, ' .82 


.70 ; 


• 59 




.73 


22 


.76 


.43 


• 49 




.50 


23 


.69 


.43 


.48' 




.4*7 


24 


.85 


.87 ■ 


.64 ' 




.97 


25 


.66 


.48 


.43 




.10 


26 


.74 • 


.61 ■ 


.46 




.72 


27 


.64^ 


. .61 . 


.41 




.45* ' 


28 


.76 


.48 


.45 




.45 


29 • 


.67 . 


-.26 


.41 




.45 


30 


.54 


.09 


.32 




.52 


31 


. -55 


.43 


• 34 




.59 


. 32 


.76 


• T61 


..46 




.72 


33 


.82 


.73 • 


.56 




.79 


34 


.67 


/ .18 


.40 




.62 




. o /. 


ad 


.40 « 




.69 ' 


36 


.61 . 


- • .23- 


" .44 ■ 




• -43 . 


37 


.67 


.55 


.46 




.50 


38 ;* 


..46 ' 


1 .36 • 


r .38 




.64 - 


39 


.45* 


.18 


.30 * 




' .52 


. 40 . 


■ .48 


, .14 


.26 




.19 v 


J" 41 


.52 


; %14 


' .41. 




.46 


42* 


-50 


. .23 


.27 


9 


.44 


43 


.53 


■ .36 


.26 




.26 


- 44, 


*.38 ' 


-.18 


.21 


c 


.3i' 


45 


• . .48 . 


.23, 


.27 . 




.27 






n ' 









TABLE 9 : ~$tem Difficulty in -Proportion of Correct 
Response for Four Reference Groups 
Mathematics Computation Subtest 



* 1 
2„ 
3 
4 
5 ' 
'6 , 
7 : 
8 
9 

18 

11 

12 

13 

14 

15 

16 

17- 

18 

19 

20 

21 

22* 

23 

24 

25 

26/ 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 



47> 
48 



CTBS/S, Level 3 



Norm Sample 



4*p 



.88 
'.71 
. .59 
.59 
.64 
.63 
.73 
.68 
.65 
.68 

.70 
.82 
.63 
.65 
.72 
.53 
.48 
.47 
.67 
.55 
.74 
.81. 
.67 
.74 
~~-^4- 
.7.0 
.81 
.64 
.64 
.62 
.62 
.69 
.70 
.58 
.69 
.69 
.54 
,52 
.65 
.52 
.65 
' .50 
.50 
.67 
.47 
:40 
.52. 



Study Sample 



\ 



V«U> 



.B7 
.65 
.41" 
.18 
.43 ■ 
.57 
.78 
.87 
.43 
.57 
.52 
.70 
.91 
, .30 
.43 
.91 
.17 
.26 
.17 
.73 
v 36 
.83 
.87 
.61 
.83 
.68 . 
.83 ' 
. { 91 
.52 
.30 
\23 
.26 
.73 
.62 
.63 
.90 
.90 
.42 
.56 
.56^ 
.47 

• 7 &. 
.35 

.41 

.76' 
.59 
• '.24 



CTBS Espanol, Level 3 



Norm Sample 



.89 
.69 
.55 
.54 
.51 
.56 
.73 
.72 
.55 
.67 
.61 
.73 
.80 
.52 
.62 
.71 
.46 
.45 
.40 
.66 
.56 
.74 
.80 
.65 
.78 
.64 
.66 
.78 
.61 
.59- 

.56 
.66 
.72 
.52 
.59 
.59 
.48 
.51 
.57 
.48 
.56 
.44' 
.45 
.59 
.47 
.34 
.48 



Study Sample 



.93 

.60 

.30 

.30 

.40 

.40* 

.60 

.70 

.47 

.47 

.43 

.67 

.87 

.40 

.53 

.63 

3 ! 



23 
.13 
.73', 
.52 
, .73 
.80 
.57 
.73 
.69 
.70 
.83 
.47 
.30 
(r . 27/ 
-'.-30 
.73 
.70 
.60 
.67 
.67 
.31. 
.43 
0' 



.77 
.33 
.36 
.79 
.25 
.39 
-.70 



TABLE 10: Item Difficulty in Proportion of Corredfc 
'Response for Four Jtef ere nee Groups: 
- r ' Mathematics Concepts and Applications Subtest/ 



/ 



CTBS/S . 


Level 3 


■ 0 , m 

CTBS ESpanol, Level *3 


Norm SamDle 


Study Sample 


Norm Sample 


Study Sample 


.84 


.95 


.87 


.73 


' .55 


.27 


? .44 . 


.23 


.82 


'.64 


.77 


.73 


-.60 


.50- 


.49 


.40, ' 


.37 


.45' 


.40 


.23 • 


.67 


.43 


.56 


.47 


.63 


.25 - 


.59 * 


.27 


.76 


.71 


.64 . • 


.50 


.51 


.27 


. ' .-41 


.43 


.58 


' .45 


•47 , 


.43 


.64 


. .36 


.60 


.40 » - 


.69 


.71 


- .64 


•57 


.65 


.67 


-•57 «, 


.47 . * 


.39 


.30 


.40 


.27 


.70 


.33 


.55 


.41 


.67 


.43 


.44 


' .47 


.73 


.67 


•6\ 
.49* ' 


.57 


.67 


.33 < - 


Jo 


' -58 


.48 


.47 


.33 


.65 


" ... .38 . 


.51 


.70 


.59 


.52 


.38 


.451 * 


.62 


.05 


,.56 • - 


• .50\ 


.74 


.73 , 


.62 * 


.73 \ 


.62 


.33 


.48- 


. -28 \ " - 


.54 


.33 


- - .44 


, .33 \ 


.60 


■ ; .62 


.68 


\ .57 


.76 


« .82^ 


.69 


' .63 


.77. 


.62 


.75 . . 


. *! .50 




.90 


.88 


.60 


.65 # 


' .52 


• .59 


' ,\ ".23 


• 66 , . 


.33 


.61 


*\ .25 


.84, 


.90 


.80 


\ .77 


.49 


' . .30 




/ .34 


' .57 


.29 \ 


.47 


,.21 


. .47 9 


.10 




-.10 '-J: 


.62 


.50 . 


.63 


.63 . 


.68 . 


.62 


• *« * .67 


*50 


' .51 


-.32 


.51 


.47 


.62 


.11 


.63 


.50' 


.66 


..32 


.60 


.57 . 


.63 ' 


.30. : 


.65 ' 1 


. .40 - 


.65 


.58. 




' .64- - . ■ 


. .48 .• 


;.2i 


• 'A* • .35 


.14 


• - . -^7-r'*" ' ' 


.68 


.62 "' 


:45 


.81- 


• .83 


.73 


.70 


.70 


.41 • 


.60 


..62 


'.'59.' , 


• .22 ' ' 


.55 


'.38 


Y6T 


. .41 " • 


.60, 


.37 • - 




.56 


.60. .* 


..41 




•:>:,;' -.-3f 

..... -34^ r 


m - ; - 50 • . * 





l^nguagte sample. This st^taialated our. curiosity and led us to 
examine ,i&ese difficult d&ems -more carefully in an. attempt to 

9 generate -hypotheses abotft why these items would be harder* for 
a Spanish-speaking group than for an English-speaking group. 

. This analysis will be presented later in this section of the < 

. report. w 

Consider now the relative performance of the CTBS Espanol 
norming sample vis a vis botji the CTBS/S norming sample and 
the CTBS Espanol study sample. This comparison is of interest 
since the CTBS Espanol norming sample was rated as bilingual- 
Illiterate. Examination of Tables 3-10 reveals that the Span- 
ish language- test was considerably easier for the LEP sample^ 
, than it was for the bilingual-biliterate sample. * This dif- ^ 
ference, though present in all subtests , was most pronounced 
on the vocabulary and reading comprehension subtests. This ■ 
• pointed to the fact that the LEP group was considerably more 4 
proficient in K the Spanish language than was the bilingual- 
biliterate group. Needless to say f given that the bilingual- 
-biliterate subjects found the CTBS Espanol to be more dif fi- 
cult than the LEP group found jLt, the item by item comparison « 
of the performance of the bilingual-biliterate group on the 
\CTBS Espanol with the performance of the norming group on -the 

GTBS/S favore&^the latter. * In other words , the bilingual- 
^ b i li t er a te^y Wup. was iess proficient in Spanish .than was the 
CTBS/S norming grpup in English. We have no information on 
the performance of tl\e bilingual-biliterate norming group on 
a the^CTBS/S. . . ' ' ^ 

\. 0 * - • % r* 

Consider now the relative performance .of one LEP group 1 
on the 5 CTBS/S and the .performance o^th€*3$fetching LEP group 
' on the CTBS Espanol (see, Tables 3-10, colufons~-2_and 4).< 
• Examination of the Tables reveals tha£/Tor^LEP students, the 
Spanish version of the test was considerably easier, than the 
English version. This was" expected. The "importance of this 
study lies r^ot in "finding" this difference but rather in 
♦documenting its^magnitude in each of the' subtests separately, 
axid in /identifying specific features of the test materials * 
•and procedures thatomight enable us to suggest ways to^ im- 
prove estimates of achievement. in LEP students. 



Igpr purposes of illustrating the magnitude of differences 
c?n the Various 'subtests within levels/ Tables 3-10 were inter- 
J preted in the form bf frequency polygons. These are pre- 
\ seiited in Figures 1-8. Figures 1 and .5 show that the CTBS * 
Espdnol Vocabulary^ subtests were considerably' easier than 
were the /CTBS/S Vocabulary subtests. "Figures 2 and 6 re- 
flect a similar relation, /for the Reading Comprehension sub- 
tests. Figures 3 and* 1 reflect no noticeable difference in 
^difficulty for title Mathematics Computation subtests. This 
.presumably reflects the *f act that language (Spanish and 
/Eng]fish) -*hal3 no effect on performance. Figures 4 and- 8 re- 
veal thatskthere probably ^re. /differences in difficulty for * 
* Level 6 1'bf fcjxe teslTput pot foh Level 3 ,for the Mathematics 
Concepts and Applications? subtests. Tests^ of these, .differ- 
ences ! will** be presented later. •* ; 
*• * 

*>, , ' : /*' h • -35-C: * Av ' . ^ - 1 
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To this point , we have established that: 

. 1. ' thef CTBS/S is easier tot English- / 
dominant students than is the CTBS 
* Espanol for Spanish-dominant students 

2. the ..CTBS Espanol is easier for Span- 
ish dominant* students than it is for 
bilingual-biliterate students 

3. -%' the CTBS/S is easier for English- 
dominant students- than is°*the CTBS • 

^ Espanol for bilihgualrbiliterate. 

4^ students. 

4. the CTBS ?spanol is easier for Spanish- , 
donjinant students' than is the CTBS/S.' 
This is more clearly the case, for- the 
vocabulary and reading' comprehension 
subtests than it is for the mathema- 
tics computation and concepts* and * , 
applications subtests. There raa^ even, 
be a difference in difficulty in the* 
latter two subtests with mathematics 
computation being ♦asier than mathe- » 

* matics confcepts and applications. 

The preceding ^xAlysis makes - clear that LES students 
encounter significant difficulties with the CTBS/S and, in 
some cases, even with the CTBS Espanol. We may now present* 
the results of an inquiry into, the specific nature of those 
difficulties. In the conduct of the inquiry, Spanish/English 
item pairs were grouped according to their levels qf ^relative 
'difficulty. The item pairs of greatest interest were (1)' 
.those an which both' members of the pair were easy (> .70**), 
(2) those in which both members of the pair were difficult 
(<£ f 30)/ and (3) those in which t&e Spanish member of the 
pair- was easier than, the English member (difference > .10) . ' 
Having identified the relevant subsets of item pairs, the 
frequency of response to each^of the response options was 
examined.* These frequencies are presented as proportion of 
frequency- in Tables 11-1.8. * The, inquiry led to a number of * 
.generalizations about response tendencies.. 

First, items in either language teftded to be easy or 

hard as a function of the 'frequency with which the key words 

are used in the respective'***languages. Thus in the vocabulary 
Subtest where items have the form: 

Given: a two to three ,word phrase where 
one of the words is underlined plus four 
single-word response options 

Required : ' select the response option 4 
that means .the .same as or nearly the same 
as . the ^underlined* word. , 



TABLE 11: 



Proportion of All Responses to* Each 
Response Option* on the Level One 
Vocabulary Subtest 



English 



Spanish 



Item 


1 


2 


3 


4 




1 


5 


•a 

-j . 


4* 


1 


.04 


.71* 


• 

• 08 


• 17 




05 

. W W 


95* 


00 

• Vw 


nn 

• U w 


2 


.23 


.32, 


.09 


.36* ■ 




.od 


.00 


03 

• WW 


97* 


3 


.57* 


-.10 


• 00 


.33 - 




90* 
• ^w 


• UJ 


n3 , 


H5 


4 


.17 


.50* 


.13 


• 21 




21 


46* 


13 

. X W 


-21 

. XX 


4 5 


.60* 


• .12 


.16 


. .12 




.62* 


08 


16 
. xo 




6 


TIT 


.50* 


.23 


• 09 




.08 


78* 
. # w 


f)3 

. WW 


• X X 


7 


.39 


.13 


.22* 


26 

. W 




23 


• X w 


96* 
• xO 


41 


8 


.09 


.50* 


.32 


09 




14 

. X*s 


57* 
• w / 


• Xfft . 


1 6 


9 


.17 


.67* 

• W » 


04 


13 




. W w 


• Ox 


n5 


• x J 


10 


.17. 


.04 

. w** «, 


17 

. X / 


63* 

• WW 




1ft 




.13 


54* 
• wfi 


ii 


* .48* 


.13 


19 


30 

• WW 




16* 

» • X w 


96 
. xw * 


n 

. XX 


47 * 
• ft / 


12' 


.4o ' 


.12. 


04 

. w-z 


.44* 




V 21 k 

\ • xx 


OS 

. WW 


f)3 

• WW 


71 * 
• / X 


13 / 


.00 


.39* 


.30 


' 30 




\l3 


79* 


18 
. xo 


nn 


14 ' 


.29 


.50* 


. 08 


13 




\?6 




• WW 


1 7 

• X w 


15 


.04 


.13 


• 22 


61* 

. WX 




ail 
. w - * 


f)3 

. WW 




87* 


16 


• 21- 


.13 


.50* 

• w w 


17 

• X # 




05 


08 

. WW 


85* 


• Uj 


17 


.04 


.08 > 


s .08 


* .80* 




.00 


03 

• WW 


13 

• X W 


8R* 

• Ow 


18 


.09 


.18 * 


• .59* * 


.14 




16 

. X W 


08 

. WW 


62* 

• wX 


1 4 


19 


.29 


.42* 


.17" 


.13 




• 08 


.74* * 


11 
« XX 


08 

. Vw 


'20 


.40 


.24 


.04 


.32* 




.26 


.03 


00 

• WW 


72** 
. / x ° 


21 


.13 


.04 


.43 


-.39* 




.00 


.08 


05 

. WW 


87* 

. W 1 • 


22 


.13 


- .17* 


.63 


.08 




.05 


.t>4* 


.18 


13 

. X w 


23 


.25 


.13 


.29* 


• 33 




.26 


.05 


18* 

. X w 


so 

• WW 


24 


.21 


.54 


.08 


,.17 




.18 


.77* 


.05 


.00 


25, 


.09 


.39 


' .26 


.26* 




.03 


.15 


05 

* WW 


77* 


26 


.4.3* 


.13 


>39 


.04 




.76* 


.16 


.05 

• WW 


.03'. 


27 


.45 % 


.32* 


.M. 


.09 




.16 


.66* 


.13 


.05 


'28 
xo 


97 

. x / 


41 




1 4 




. u J 


• Xw 




nc 

.uo ■ 


29 


.10 


.10 


.40* • 


,*33 




.03 


.11- 


.54* . 


..32 * 


30" 


..64 


• .14 " 


.0B* 


.18 




.62 


-.05 


.18* 


.15 


-31 : 


.32 


- ;14 


• <75. 


.50* 




'..02 . . 


.05 


.05 


.82* . 


.. 32 


.33 


.29 


.05 


.33* 




-.46 


.00 


.10 


.44* 


33 


.25 ' 


.35 


, .20* 


.20 




.21 


. -28 


.13* 


«38 


34 


. .14 . 


.29 


.33* 


.24 




.11 


.08 


.59* 


.22 




.24 


.33 


.00* 


• .■ .43 . 




.24 


. .16 


.45* 


.16 


• 36 


.36* 


;27 . 


. .23 


.14- 




.76* 


.08 


.03 


.14 


37 * 


.19* 


■ .48 


.19 m 


► .14 




.49* 


.14 . 


.11 


.26 ■ 


38 


...27 • 


.36 


• .09 


.27* ' 




.17 


.54 


.14 


.14* • 


39 


.52 


.••24 . 


.24* 


.00 




.17 


.28 


.42*^ 


1 .14 


40 


• .05 


'\43 


.14 


.38* 




.20 


.11 


.14 W\ 


.54* 



^Indicates correct answer 



TABLE 12: Proportion of All Responses to Each 
. Response Option on the Level 'One 
Reading Comprehension Subtest 
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ERJC 



English 



Spanish 



.Item 


* 1 


. 2 


3- 


4 




1 


2 




w 4 


i. 


.16 


.08 


.20 


.56* 




.08- 


: .oo 


.16 


>7.6* 


' 2 


.52* 


.13 


.04 


.30 




.79* 


.00 " 


.08 "* 


.13 


3 


.32 


.20 


.16 


,*32*« 




-.08 


.08 


. .08 


. .76V 


4 


.32 


.44* 


.12 


.12 




.18 ' 


.79* 


.00 


.03, 


'5 


.08 


.12 


.16 


.64* 
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- - .Gf5 . 


-.03 - 


.9-2* 


6 
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.72* 


• ,04 
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* 


.44 


- .73* 


.05 


.08 


7 


.13 


.17 


.13 


.58* ' 




; .00 


. -19 


.08 


.73* 


8 


.13 


.79* 


.04 


.04 




.32 


.62* 


'.03 ' • 


.03 


9 


.18 


.36 


.18*. 


.27 , 




-.11 


.27 


°.38'*; 


.24 


10 


.00 


-.33* 


.21 


.46 




.00 


„73* 
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.16 


-11 . ... 


•13* . 


• 46* 


.08 


.25. . 




L .13 
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- .05 " 
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12 


' .21 


, .46* 


' >p8 ; 


.25 




*.13 


.47* 


.05. 
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13 


.26 


.22 


J17 • 


* .35* 


• 


.06 '• • 


..08 


.19' 


.67* 


14 


. .29* 


.17 


.25 ' 


.29 . * 




.45* 


' .34 ' 


.03 „ 


.18 ' 


15 ■ 
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.13 ' 


.42 


.17* ' 




.03 . 
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16 


.22 
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.65* 
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.13 
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.68*. 


.13* 


17 
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.25* 
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.00 
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% .00 


18 
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♦ .27' 
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19 


.09 


.30 


■ .35 
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.11 


.30 


.24 
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. .13* 


.22 


.30 
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.29* 


.05 


.21 


,45 


21 
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* .36 


.20 
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.18 


.08 " 


.11 


22 


.17 


.21 
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.42* 
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.23 . 


.11 ' 


.51* 


23 


.08 


.13 


.38* 


.42 




.03 
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24 

* 


.08. 
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.19 
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• 


.11 
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-.08 ■ 
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.00 
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.00 
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28 
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.00 
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, .12 
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.13 ,. 
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&.09 


31 
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r 
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.66* 


32 
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.28* 


.44 
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.00 
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.12 


33 


.08 . 


.17 
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.23 


.11 


.63* 
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.04 


.67* 


.17 


.13 




.03" " 


,66* 
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.20 


35 


• .04 


.38 


.38* 


.21 , 




.03 


.20 


.71* 


.06 


36 


.17 


.17 . 


.09 


. .57^ 




.23 


.17 
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.49* 


37 


;17 _ 


.04 


. .71* 


,08 




.03 


.06 


.74* 


. .17 


38 


.09 


.39* ' 


.35 


.17 




.17 


.60* 


.11 ." 
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.29* 


. .21 


.21 


.29 




.24* 


.41, 


.18 


.18 


40 


.17* 


.17 


.22 


.43 




.54* 


.14 


.23 


.09 


41 


.04 


.13 


.74* 


.09 




.06 


.12 


. .73* 
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42 


' .12 


.28 


,28 


.32* 




.26 


.17 ■ 


.11 


.46* 


43 ... 


.13 


.33 


.21 


.33* 




.14 
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.09 


.51* 
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.13 . 


;17 
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.18 
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.24* 


45 
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.09 


.39* 




.23 • 
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.09 


.63* 



^Indicates 



cbrsept, gnawer' 



TABLE 13: Proportion of All Response to Each 
Response Option on the Level One 
: tf* Mathematics Computation Subtest 



\ 



. - 


English 


* 




Spanish 




• 1 


2 


3 


4 




1 ' 


2, 


3 


4 


no 


1,00* 


.00 


.00 




.00 


i 00* 
X . uu 


OO ' 
• uu 


.00 




.96* 


.00 


• 00 




.00 


i 00* 

X . UU" 


00 
• uu 


.00 


no 
• uu 


.92* 


• 00 


.00 




.00 > 


°1 • OO* 

X • uu* 


00 
• uu 


-.00 


1_ft 


.QjO 


.00 M 


*" O A * 

• 84* 




.11 


• UJ 


00 
• uu 


* ,87* 


OP. 
• uo 


.88* 


.00 


.04 




.08 


• , 


OO 

• uu 


.00 


nn 

. WW 


.21 


-A 

• 75* 


• 04 




.06 


no 
• 08 


.92* m 


.00 


i • u** 


.12 


.08 


.76* 




,.00 


• 13 . 


, .00 


.87* 


. 0<* 


.04 


^.88* . 


f\A 

.04 • 




.00 


0*3 
• U O 


Q7* 


.00 


..04 


.13 


t 75* 


no 
.08 




.03 >» 


OO 

• uu 


Q7* 


.05 - 


• U 4 * 


:04 


.33* 


no 




.14 


• XO 


7fl* 
• /o 


.00 


ftp.* 
_ . DO 


.20 


• 

• 00* 


• 12 




.92* 


n*3 
• Uj 


• J J 


• 03 


00 


• 08 


.38 


.54* 




.06 


Oft 

. uo 


• X*t 


% ,7S* 


OP. «* 
• UO 


.75* 


• 13 


• 04 




.03 p 


Q7* ' 


00© 


• .00 


• jO 


• 00 




• 16 




.84* 


* nn 
• UU 


• XX 


.95 


00 

• uu 


.32 


• 00 


*63* 




.00 


• XX 


OO 
• UU 


.89* ■ 


AO 
• UU 


.08 


•92* 


• 00 




.00 * 


00 
• uu 


Q7* 


.03 


.uo 


.88* 


• 04. 


• 00 




.05 


fib,* 


OO 
• UU 


.05 


i 7 
• X* 


.76* 


.08 


V> A 




.03 


• • 0 /* 


05 


.05 - 


oa 
. 2 / 


.72* 
.05 


>23 


1*5* 




.03 

11 


no 
• Uo 


OO 

• uu 

• X4 


.^08 
.67* • 


• UU 


.04 - 


• 96* 


• 00 


— 


* .00 


0*3 


QQ* 


.03 


• X / 


.00 t 


.83* 


•00 




.18 


. Uo • 


7ft* 


.03 


1 3 
• J. J 


. ^17 - 


.22 


.48*" 




.05 


.11 


• Xo 


.68* 


nn 
• UU 


:i7 


.52* • 


.30 




.08 


1 7 

• X / 


ftA* 
, . 0** 


.11 


nr\ 
• UU 


'1.00* 


.00 


♦ .00 




<v05 


AC* 


OO 

- . uu 


.do 


nn & 
- • UU 


1.00* 


.00 


• 00 




.00 


i no* 
X • uu 


OO 

• UU 


.00 




.08 


.00 


» *op 




• .02* 


AO 

. Uo 


OO 

• uu 


.00 * 


• XO 


.00 


.88* 


.04 




. .05 


*^ OO 

• uu 


QS* 


.00 


nn 

• UU .. 


.08 ' 


.08 


•83* 




.03 


. Uo 


OO 

. uu 


.89* 




.04 


• 08 


• 42 




.78* 


rtC 

• 05 


OO 

. uu 


ae 


6 n/i 
• U4 


.25 


.63* . 


• 08 




.03 


• X*4 




.05 


nn 
• UU 


• 04 


.96* 


• 00 




' .03 


• UO 


.07 


.00 


1 7 


•58* , 


.08 


•17 




•16 


70* 
. / u 


• Uj 


.11 


no. 
• k Uw % ^ 


.00 


. .92* 


.00 




• 08 


OO 
• UU 




. .00 


OA 

• u«* 


• 21 


.04 j 
.25^ 


• 7J. W 




.05 


1 ft ' 

• XO 




• 76* 


OA 
• U«* 


.08 . 


.63* 




• 08 


.05 


.11 


• 76* 


00 . 

• UU * 


.00 > 


.92* 


.08 




.00 0 


.03 


• 89* 


• 08 


.08 


• 00 ~ 


.83*. 


.08 




.03 


.00 


.95* 


• 03. 


.24 


.76* 


.00 


.00 




".16 


.84* 


Too 


• 00 


.92* 


.00 


\04. - 


.04 




.92* 


.03 


.00 


.05 


.08 


;88* 


• 04 


.00 




.05 


.86* 


• 05 


• 03 


.88* 


.00 


.00 


.12 




;86* 
.86-** " 


.03 


• 00 • 


• 10 


.88* 


.04 , 


.04 ~ 


.04 




, .08 


.00 


• 05 


'.'00' 


..78* 


.04 ■ * 


.17 




.00 


.84* ' 


♦.08 


• 08 


.83- 


.00 ■ ■ 


, .08 


• .08 




- .84* 


.05, 


.08 


.03 


.13 


.17 


' .33 


, '.38* 




.14 


.22* 


.27 


' .38* 


.26* 


.17 


..22 


.35 




.43* 


.14 


.08 


• 35 


.13 


.42* 


• 25 


.21 




' . Ur 

c 


.55** 


.18 


.16 



atas correct answer - -47- :\ §\ 



TABLE 14; Proportion of all Responses to Each. 

Response Option on the Level One * * 
Mathematics. Concepts and Applications Subtest 



• 


• 


English 






Spanish 




Item 


1 


2 


3 


•4 
* 




1 


2 


3 . 


4 


1, 


on * 


.24 


♦ 24 


• 24 




• Zft w 


.30 


• 22 


• Z4 


ft 

z 


n>i 
.04 


.12 


.00 


.84* 




no 
• Uo 


• 03 


• 00 


ftO* 
• oy* 


o 

3 * 


• UU 


.00 


• 88* 


* to 
.13 




no. 
. uo 


.00 


• .89* 


n*3 
• uo 


A 


• lo 


.00 


.76* 


• 08 




• 11 


.05 


.84* 


nn 
• uu 


c 


HQ 
• Uo 


.88* 


.00 


• 04 




An** 

• UU 


.97* 


• 03 


nn 


D 


1 £ 

• lo 


.64* 


• 08 


.12 




n^ 

• U3 


0 1 <fr 

.87* 


• 03 


nt% 

. «U3 


/ 


fin** 
• ou 


• 20 


.16 


.04 




• / o 


• 11 


• 03 


nft 
• uo 


» ft 
' O 


• DO ' 


.24 


• 08 " 


.12 




— 7Q*' 


• 03 


• 05 


• lo 






Q 

9 # 


nn 

• UU 


.04 


• 88* 


* .08 




n^ 
. uo 


.05 


.92* 


nn 
• uu 


i n 
lu 




.1-2 


• 80 


.04 




• li 


.11 


.76* 


• uo 


* 11 


nn 
• UU 


.88 


.12 


.00 




nn 
• UU 


5^ 


.05 


»nn » 
• uu * 


1Z 


eo * 
• o3 x 


.08- 


>08 


.21 




.ol 


rw 


.05 


nft 
• uo 


i o 
13 


• lo 


.45* 


• 05 


• 32 




no 
• Uo 


.53* 


.00 


• or 


14 


1 o 

.13 


. .04. 


"' ,.43 


/ .39* 




• .11 


.03 . 


.47^ 


*3 O* 

..39* 


1 c 
13 


• 13 


.09 


.43* 


.35 




nft 
• Uo 


• .t)5 


^29* 


• JO 


lb 


• 16 


.48* 


• 32 


, .04 




nn • 
» UU 


.71* . 


.24 


• UO 


1 / 


• 16 


.12 


.48* 


.24 




n*a 
• Uo 


.24 


.73* 


nn 
• UU 


lo 


.04 


.04 , 


.00 


.92* 




nn 
• UU 


.00 


.03 


Q7* 


19 


• 09 


.35* 


..17 


.39 




* nn 
• UU 


.50* 


.13 




ftft 

20 


• 0(r 


.79* 


.17 


.04 




nn 
• UU 


.92* 


.03 


• UO 


oi 
21 


ft ft 

• 00 


.44* 


. .40 


.16 




nn 
• UU 


.82* 


.16 


* n*3 

• uo 


oo 

22 


O/l 

• 24 


.12 


.36* 


.28 




no 
■ \ .Uo 


.06 


.56* 


• Ol * 


23 


.17* 


.21 


.25 


.38 




Adit 

• 4o x 


.24 


.08 




24 


.14* 


.33 


.33 


.19 




vt 0 * 

• 43* 


.14 


i .26 


1 7 
• 1 / 


25 


o e 

• 25 


.50*' 


.17 


.Q8 




.11 


.63* 


.05 


Ol 
• Zl 


26 


T ft 

„12 


.12 


. .76* 


.00 




• 15 


.15 


.76* % 


. .13 




.54* 


.17 


.08 


.21 




Q O * 
• OZ* 
* 


.03 


.05 


• 11 


28 


.54* 


.08 ' 


.29 


.08 




/IE* 


.08 


• 45 


**** • 03 


29 


• 08 


.17* 


.67 


.08 




• 11 


.65* 


.14 


• 11 


Oft 

30 


, • 22 


,.57* 


.04 


.17 




1 ft 

• .lo . 


.53* 


.11 


1 ft 

• io 


31 


ftp 
• 05 


..36 


*.59* ( 


.00 




• 13 


.26 


.61* 


• 11 


"3 O 
3Z 


I • 3U X 


.30 


.09 


.30 


- 


AO* 


.14 


. .14 


' 1 1 

• XX 


"3 O 

33 


• 3o 


.00 


• 33* 


.29 




"3ft 

• OO 


. .12/ 


.35* 


• xo 


Ovl 
34 


• OU 


.15* 


.15 


• 20 




• 31 


o.' .35* 


fto 
.Op 


n5 


35 


ns 
.UO 


.35 


.20 


.40 




n£ 
• uo 


.57* 


• 20 


1 7 

• X / 


3d 


• ZD 


-.22 


.13* 


.39 




n^ 

• U3 


• 32 


.35* 


97 


3 / 


• UU 


.41 • 


.56* 


• 09 




n^ 


, .16 


CO* 

.63* 


1 

. xo 


oo 
38 


/fa * 
x .43* 


• 22 


.09 


la 1 




47* 


1 O 

.19 


ft ^ 

.06 * 


Oft 


oo 

39 


• .U9 


.18* 


.55 


.18 




nft 
• uo 


.4 7* 


OC 

• • 25 


• X -7 


40 


oc 
• 2o 


• 47* 


- .16 


.11 




n£ 
• uo 


• OD* 


*\A ' 

• z4 


nfi 

• uo 


41 


• 3y^ 


AT. 


1 7 
• X /, 


nn 




7n* 
• / u 




16 




AO 
4Z 


n4 

• Ufi 


.48* 


.39 


.09 ' 






.58* 


.21 


16 


43 


05 


.59* 


. .14 


.23 




.16 


.43* 


.19 . 


.22 


44 


.08 


. .46* 


.17 


.29 




.08 


.50* 


.24 


.18 


45 


.09 


.43 


.35* 


.13. 




.11 


.03 - 


.58* 


.28 


46 '. , 


.35 . 


.17* 


.13 


.35 




7.26 


,.45** 


.03 


.26 


47 


;i3 


. .65* 


. %13. 


«09 




.18 


.63* 


..11 ~ 


.08 


48 


. -43 


.'35 


.22* 


.00 ' 




.27 


.38 


.32* 


.03 


49 


.38 


.43* 


• .14 


" .05 




.14 


.76* 


.03 


.08 


50 . . 


; 74 2 * ; " 


.25 


.13%, 


;2i . 




■ . : 41*- 


v24 


.19 


.16 




s,:correct. 




% -4P- 


62 








* 





TABLE 15: Proportion of All Responses to Each 
Response Option" on the Level Three 
Vocabulary Subtest * 





— : 


English > 








Spanish 




Item 

• 


1 


2 


3 


4 




1 


2 


3 


4 


1 


• .43* . 


t 

,17 


.04 


• '.35 


• 


t 

.67'* 


.13 


.00 


.21 


2 


.35 


.43* 


.09 


.13 




.29 . 


.46* 


.08 


.17 


3 


.26 


.43* 


.26 


.04 . 


* 


.25 


.38* 


.33 


.04 


4 


.39* 


.22 


.13 


.26 




4 .58* 


.08 


.04 


-.29 


• 5 


.26 


.13 ' 


.52* 


' .09 




.13 


.08 • 


.79* 


.00 


. 6 


.26 


.39* 


.22 


•L 


3 




' .00 


1.00* 


.00 


.00 


' 7 


.22 


.17 ' 


' .39 


.2 


2* 




.13 


.00 


.13 


.75* 


8 


. 26 


.30* 


.13 


.3 






.13 


.63* 


.04 


.21 


9 


.13 


.61* 


..04 


.2 






. .04, ' 


.92* 


.00 


.04 


10 


.30 


.43 


\04« 


•5*5* * 




" .13 • . 


. -33. 


.04 


.50* 


11 


.13 \ 


.78* 


.09 


. UU 




.08 


.75* 


.13 


.04 


'12 


.13 


.13 


.35 






.04 


.04 


.04 


1*88* , 


13 


.35 


.22 


.35* 


no 




.13 


.04 . 


.75* 


.08 


14 -"t 


.04 


.30 


.61* 


. U4 




.21 . 


.08 


.71* 


.00 


15 


.80* 


.04 


.04 


.22 




.75* 


.13 • 


.04 


" .08 


16 


.04 


.17 


.13 


.65.* 




.00 


.08 


.00 


.92* • 


^7 


.22* 


.43 , 


.09 


.26 . 




.96* 


.00 


.00 


.04 


18 


730 


.00 


.43* 


.26 




.96* 


.00 . 


.00 


.04 


19 


.65 


.09 


.09 


.17* 




.17 


.08 


.00 - 


.75* 


20 


.13 . 


.43 


.35* 


.09 




.08 


.17 


.67* 


.08 


21 


.30* 


.26 


.43 ' 


.00 




.63* 


.17 


.17 


.04 - 


22 ' 


.35 . 


.26 


.09 


.30 




.88* 


.08 


-.04 


.00 


23 ~ 


.30 


.17 


.43* 


.09 




.38 


.42 - 


.21 


.00 


24 


.35* 


.35 


.26 


.04 




'.58* 


.29 - 


.13 


. -00 


25 


.27 


.18 


.32* 


.23 




.17 


.79, 


.00* 


.04 


• 26 


.32 


.14 


.14 


.41* 




-.33 


.13 


.00 


.54* 


27 


.59* 


.0* 


.14 


.18 




.71* 


.17 


.08 


.04. 


23" 


,32 ' 


T8 
• io 


23 


.27 




.42 


.25* 


13 


.21 


2 ?- 




.14 


.38 


.33* 




.04 


.08 • 


.25 


.63 


30 - 


.62 • 


.10 


.29* 


.00 




.50 


' .17 


.21 


.13 


31 


.14 


' .67* 


.0.5 


. .14" 




.13 


.75* 


.04 


,.0*8 


32.. 


.15 


.33 


.38* 


.24 




.08 


.08' 


• - .■&!* 


.17 " 


33- - 
'34 


* .38V- - 


- .10" 


.10 


.43 




.29 ' 


• .13 


.04 


.54* 


" ■ .56* 


.10 


.24 ' 


.10 




.48* 


.04 


.22' 


.26 


35 


.14 


.38 


.10 


.38 




.09 


.ft 


.00 


.59 


36 


» .25 


.10 


>.20 


.45 




.14 


. .09 


.14 


. .64 


37 


.11 


.42* 


.37 


.11 




..14 


.'36* 


.41 


.09 


38 


.16 


.37 


.26 


.21* 




.23 


.27 


.14 


.36* 


39. 


.47* 


.21 


.05 


.26 




.68*- 


.18 


.09 


.05 


40 


.32 


.21* 


.32* 


%16 


% 


.04 


.87*" 


.00 


.09 



♦Indicates correct .answer 



TABLE 16: Proportion o|? All Responses to Each ' • 
Response Option on the Level Three 
Reading Comprehension Subtest 







English 






Spanish ' 




- Item 


1 


2 " 


3 


4 




1 


2 


3 


( V 


1 


.70* 


.13 


'.09 


• 

.09 




.8Q* 


.13 ; 


.00 


.00 


2 


• 22 


• .26 


.35* 


.17 




.04' 


.04 


.88* 


.04 . 


3 


.35 


.22 


.22* 


.22 




.04 


.13 


.75* 


, .08. 


.4 


' -39 


.39* 


.17 


.04 




. .08 


.63* 


.17 


.13* v 


5 


.22 


.39 


.39* 


.00 




.54 


.29 


.17* 


.00 


6 




.14 


.59* 


.27 




.04 


.04 


.43* 


.48 


7 


.18 


.18 ■ 


.23* 


.41 




.13 


.08 


.50* 


' .29. 


8 


.35 ' 


.17 


.35 


*13* 




.13 


.21 - 


.17 


.50* 


9 


.35 


.17 


.35 


.13* 




.13 


-.21 


.17 

.08' 


.50* 


10 


.17* 


.26 


.13 


.43 




.29* 


.08 


.54 . 


11 


.04 


.83* 


" .04 


.09 




.04 


.92* 


.00 


.04 


12 


.22 


.48* 


.13. 


.17 




.04- y 


.65* 


.13 


.17 


13 


.52 


.17 


.17 


.13* 




.08 


.04 


.25 


.63* 


14 


.39* 


.26 


. .22 ' 


.13 




.50* 


.08 


.08 


.33 


15 


.13 


.13 


V .52 


.22* 




.23 


.13 


.27 


.37* 


16 


.09 


.35 


.48* 


.09 




.23 


.23 


.30* 


.23 


17 


.43 


.39* 


.00 


.17 




.17°" 


.63* 


.17 


.03 ' 


18 


.26 


.70*. 


.30 


- .00 




.07 


.73* 


.'17 


.03 ' 


; 19 - 


.30 


.22 


.17 ' 


.30* 




.13 


.13 


. .13 


.60* 


20 ■ 


.09 


.52 


- %-35* 


.04 




.00 ■* 


.10 


.77* 


113 


21 


*00 


.70* 


.30 


' .00 




~.07 


.73* 


.17 


.03 . 


22 


.30 


.09 


..17 


.43* 




.20 


,17 


.13 


.50* 


23 


.43* 


.04 


.13 • 


.39 




" .47* • 


..07 


.47 


.00 


24 * 


.09 


.87* 


.00 


.04 




.03 


.*97* 


.00 


• 00 


25 


.22 


«17 


.13 


.48* 




.21 


'•> .31 


.38 


.10* 


» 

' 26 


.22 


,09 


.09 


.61* 




' .07 


.17 


.03 


.72* 


21 


.09 


.17 


.13 


.61* 




.28 


.14 


.14 


.45*. 


28 


.13 


.22 


.48* 


.17 




.28 


.10 


. .45* 


.17 


29 


.26* 


-.43 


.22 


.09 




/ 45* 
.52* 


.24 


.21 ' 


.10 


30 


.09* 


.39 


.17 


.35 




.07 


.14 


.28 


31 


.00 


.22 


.35 


.43* 




.07 


.10 


.24 


.59* 


32' 


.17 . 


.09 


.13 . 


.61* 




..03 


.14 


.10 


.72* 


33 


.73* 


.23 


,do_ 


.05 




.79* 


.10 


.10 


* .00 


34 


.36 


.18 


.27 


.18* 




.17 


.17 


.03 


.62* 


35 


.09 


.05 


.23 


.64* 




.17 


.07 ? 


.07 


.69* 


3.6 


.18 


.23* 


.45 


.14 






.43* 


.21 


.11 , » 


*37 


.00 


.55* . 


.36. 


.09 




* .04 


.50* 


.36 


.11 * 


38 


.18 


.36* 


.09 


.36 




.11 


.64* 


.18 


,07 


39 


.50 


.23 


.09 


- .18* 




.19 


.19 


.11 


.52* 


. 40 


* .,14* 


• U5. 


.14 


.27 




'.19* 


.37 


.17 - 


• 26 


41 


.23 


.14* 


.55 


4 Q9 




.04 


.46*, 


■ .08 


.42 


' 42 


.23* 


.36 


.05 


-.36 ' 




.44* 


.11 


.22 


.22 


■ 43- 


.27 


.27 


.09 


• .36* 




• .41 


.22 




.26* 


44 


. .18* 


.32 


.32 


' .18 




.31* 


—^31 




.19 • 


45 


.36 


.27 


.14 ' 


.23* 




.23' 


.18* 


.32 


,*.27* 



^Indicates correct -answer 



TABLE 17: Proportion of All Responses to Each 
Response Option -on the Level Three 
■ Mathematics Computation Subtest 



• • 




English 






Spanish 




Item 


1 


2 


. 3 


4 




1 


2 


3 


4 ' 


1 


.87* 


.04 


.09 


.00 




.93* 


.00 


.00 


.07 


2 


.26 


.09 ' 


.00 


.65* 


• 


.10 


.27 


.03 


.60* 


3 


.41* 


.23 


.32 , 


.05 




.30* 


.17 


.40 


.13 


4 


.09 


.41 


.18* 


.32 




• .10 


.23 . 


.30* 


.37 


5 


* .17 


.39 


.43* 


.00 




.10 


.27 


.40* 


.23 


6 * 


.26 


.04 


.13 


.57* 




.47 


.03 


.10 


.40* 


7 


- .17 


.78* 


.04 


.00 




.20 


.60* 


.17 


.03 


8 


.13 


.87* * 


.00 


.00 




.20 


.70* 


.07 


.03 


9 


.09 


,39 


. .43* 


. .09 




.20 


.13 


.47* 


.20 


10 


.26 


.57* 


.09 


.09 




.27 


.47* 


.20 ... 


.07 


11 


.17 


.09 


.22 


' .52* 




.17 


.27 


.13 • 


.43* 


12 


.70* 


.09 


.13 


".09 




.67* 


.17 


.10 


.07 


13 


.04 


.91* 


.04 


.00 




.07 


• .87* 


.03 


;03 


14 


.17 


.30* ' 


.30 


.22 




.10 


.40* 


.27 


.23 


15 


% . .43* 


.36 


.22 


.04 




.07 


.07 


.63* 


.23 


( 16 


.00 


.04 


.91* 


' .04 




.07 


.07 


.63* . 


.23 


17 


.26 


.43 


.17* 


.13 




.17 


.53 


.30* 


.00 


18 


.13 


.26* 


.35 


.26 




.10 


.23* 


• 3,7 


.30' 


19 


.09 


.17* 


.74 


.00 




.03 


„13* 


.80 


.03 


20 


.05 


.09 . 


.73* 


.1*4 




.07 


.. .10" 


.73* • 


.10*' 


21 


.09 


.36 


.18 


.36* 




.21 


.17 


.10 


.52* 


22 


.00 


.83* 


.13 


.04 




.00 


.73* 


.10* 


.17 


23 


.04 


.00 


.87* 


, -09 




.03 * 


.07 


.80*'. 


.10 


24 


.61* 


.04 


• .30 


.04 




. .57* 


.13 


.20 


. .10 


25 


.04 


.09-* 


.04 


.83* 




.03 


.20 


.03 


.73* 


26 


.05 


.18 


..09 


.68* 




.03 * 


.17 


.10 


.69* ' 


27 


.09 


;83* 


, .00 


.09 




.10 ' 


.70* 


.10 


.10 


28 


.91* 


.09 


.00 


.00 




.83* • 


1 .10 


.07 


.00- 


29 


.52* 


.22 


.22 


.04 • 




^47* 


.30 


.23 


.00 


30 


.04 


.30 


.61 


.04 




.07 


. .30* 


.43 


.20 - 


31 


.23* 


/ .00 


.27 


.50- 




.27* 


.03 


-.40 


.30 


32 


.26* 


-.13 


.52 


.09 




.30* 


.27 


.40 


.03 


33 


.05 


.09 


' .73* 


.14 




.07 


.07 


.73* 


.13 




. .05^ 




• .14 


.62* 




.07 


.10 


.13. 


.70* 


35 


.21 ' * # 


.63* 


.11 


.05 




.07 


.60* 


' ;30 


^03 


36 


.11* 


• .00 


.16 


.74* 




.10 


-.07 


.03 


•80* 


37 


.05 


.90* 


.00 


.05 




.10 


.67* 


.03 


.20 - 


38 


.11 . 


• .11 


.42* 


.37 




.03 


.14 


.30* 


.52 


39 


.00 • 


.17 


.28 


.56* 




;o7 


.18 


.32 


.43* 


40 


.17 (V 


- .28 


.00 


.56* 




.17 


.17 


.17 


.50* 


*41 


.11 


.32 


.11 


.47* 




.10 


.60. 


' .07 


r .23* 
,03 


42 


.12 


.00 


.76* 


- .12 




.10 


.10 


.77* 


43 


.18, " 


.29 


.18 ' 


.35*. 




.20 


.40 


. .07 


.33* 


44 


.41* 


.*29 


.06 


.24 




.36* 


.14 


.18 


.32 


45 


.12 


.76* 


.12 


• .00 




.04' 


.79* 


.07 


• .11 


♦ 46 




.12 


.24 


.06 




.25* 


.18 


.50 


.07 


47 


.24 * 


.35 


.18^ 


.24 . 




.39* 


". .25 


-.14 


.21 . 


^ 48 m 


.00 


.69* 


.06" 


.25. 




.07 


• 70*„ 


.15 


.07 ' 








n w 










* • 



♦indicates^ .correct answer 
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TABLE 18: Proportion of All Reafponses to Each 

Response Option on the L3vel Three 
" * * 9 Mathematics Concepts and Applications Subtest 



1 ERJC 



• 




English 




• 


Spanish 


fl 


Item 


* ' 1 


2 


3. 


> 4 




•1 / 


2 


3 - ' 


4 ™ 


v 

1 


% .05 


♦ 

.9.5*. 


.00 


.00 




.03 


- .73* 


.10 


.13 


2 


. .05 


.27* 


.32 • . 


.36 




• 20 


.23 - 


.33 • 


.23 


3 


.00 


.09 


'.27 


3 .64* 




• 03 


.17 


..07 


,.73* r 


4 


•.00 • 


. -41. 


.09 


.50* 




• 03 - 


.20 


.37 


.40* 


5 , 


.45* ' 


.18, 


.27 


.09 




.23* . 


.27 


.40 , 


.10 


6 * 


- .38 


,.14 


.43* 


.05 




• 40 


.27 


.27* 


.07- 


7 


.40 . 


• .30 ' 


.25* 


.05 




\40 


.fl 


,.27* • 


.07 


8 


.71* ' 


.00 


.14 


.14 


* 


> 

• 50* 


« 

< .07 


.37 


, '.07 


9 


.27* 


.23 


.09 


.41 




.43* 


.17* 


' 2 i 


• 13 


10 


.45* 


.18 


.09 . , 


.27 




• 43* 


.17 t 


.27 


••13 


11 


.36* 


.27 


.27 


.09 




• 40* 




.23 


'•16 - 


12 


.14 


.05 


.71* 


.10 




• 10 


.17 


.57* 


.17 


13 J, 


.*67* 


.05 - 


.14. 


.14 




.47* 


. '.10 


' .23 


\*20 


14 


.30* 


.15 


.30 


.25 




.27* 


.17 ' 


.37 


• 20^ 


15 


.14 


.38 


.33* 


• 14 . , 




.17 . 


.31 


"U'l* 


• 10* * 


16 


.19 


.14 


.43* 


' .24 




.23 


.10 


$7* 


.20 


17 


.67* 


.10 


.19 


.05 




.57* 


. .30 


.07 


.07 


18. 


.10 


.33* 


.52 


.05 




.07 


.-40* 


.40 . 


.13 


19 


.05 


.33 


ft* 
.38* 


.48* 




• 33 


.20 


..13 


.33 \ 


20 


.14 . 


* .33 


.14 




• 0> 


•17 


.70*\ 


.07 ^ 


21 


-.00 


.52* 


-.19 


.29 




.21 


#.45* 


.14 


• 21 


22 


.14 


.38 


. .43 


.05* 




.17 


i * 23 


,.10 


• 50*^ 


23 


.14 


.73* 


.05 . 


. .09 




.20 


.* .73* 


.02 




. 24 


.14" 


.24 


.33* 


.29 




.24 


• 31 s 


.28* 




25 


.43 


.10 


.33* 


.14 




.40 


.13 


.33V 


^•13 - 


26 


" .05 


.14 


-.19 . 


..62*, 




.13 


.10' 


.20 ,> 
.27 


.57* 


27 


.82* 


.00- 


' .14 


.05 




. .63* 


.03 


• 07 


28 


.10 


.29 ' 


.'62* 


• .00 




.27 


.17 


.50* 


.07 


'29 


.05 - 


.90* 


.05 


.00 




.07* 


.60 




.07 


30 


w k 


.52* 


.24 ' 


.14 




.23* 


* .23 


/% 


.27 


31 


.33* 


.24 


.33 


.10 




.25 




* .43 


" .18 


32 


.90* 


.10 


.00 


.00 • 




.77* 


-.13 


.03 


.07 


33- 


^s.35 


.30* 


.25 






.45 


.34* ' 


xXO 


.10 


34 


. T48 


.14 


. .29* • 


.10 




.,59 


<*clb 


-,.21* 


.10 


3.5 


.10 - 


.29 


.10* 


* .52 « 




.07 


.40 


' .10 ♦ 




36 


' .25 


.10 


.15 


.50* . 




.20 


.07^ 


.10 


^.63* * 


37 


.62* 


.05 


.24 


.10' 




-.50* 


.10- 


.10 


.30 


38 . 


. .05 


.32* 


.16 


.47 




*L0 


.47* 




.33 


39 \ 


• " .05 


.79 


.05 


.11* 




.00 


.43 


.07 


.50* 


'40 


\ .11 


. .32* 


.26 


.32 • 




« .03 


.57*' 


.23 


.17. 


41 


.35 


.20 


. ~15' 


' .30* 




.23 


.17 


.20 


• 40* # 


42 , ' 


* .16 


.11 


'.58* 


'.16 




.04 




.64* 


• 21 


43 


.42 


.32 . 


.21* 


,05 




.59 , 


.28 


.14* * 




44 


.11 


.68* 


.-21 . 


.00 




.34 


.45* 


.14 


.01 


.45 


.06 : 


".83* 


.06 


.06' 




-.00 


.70* • 


.13 


.17 


46 


.06 


• 35 


.18 


.41* 




M0 • 


.14/'-' 


J.4- 


.62* 


47 




.22 


.22* 


.44 




.10 


.14 


.38-* 


.38 


48%. • ' 


.00 


.41 


.41* 


.18 




.10 „ 


•43 . 


-.37* 


.10 


if. 


.25 


;56* ' 


.06 


".13 




-.17,- • 


.41*" 


.17 


.24 , 


50. 


. .ir 


.31%-. 


V -3|^ 


.17 




.33 


.30* 


.27 


.10 , 






: ' . *- *• 

















ataa correct: answer. 



students" found* it easy ' t;o o matqh , for example r casa to residen 
cja knd house, * to residence , 

— — ~ - »- * », 

, This particular example also points up another- feature 
of several of the easiest items , namely*, the presence of 
cognates. Examples of such pairs in the vocabulary -subtest 
(item difficulties in- parentheses) , are-: • • - 

«? - . * 

. 1. equivalent/equal • * . .(•78)- > 0 , 
eqtjivalente/iguales* (..75) , - 

. 2. affirmative/positive ^(.67) 

af irmativoV^ositivq '* ( • 7 5 ) V ° 

— ■ • ^ ^ v * ' ' • ' V r * 

Such items were not always .easy. Sometimes *they fell in a. 
mid-range of difficulty^ Whrenever^this happened \ the level* 
of difficulty w£s about the sa$e for each language.* Examples 



are: 



'ly' dialogue/conversation * ( . 41) 

di^iogoVsbnversacion (.54) 

2. . confirmed/ verified . (.43)" 

cbnf irmada/v^rif icada ( . 54 ) 




--^^xaagt ^cases o£ cognate pairs-f the Spanish language iteffTwas 
, somewhatf^^sier^tjhan the English language^fesp* * 



The items discussed above may be referred to as "pure" 
cognate pairs since s both stimuli and responses are cognates. 
There were -also examples of "partial" cognate pairs where 
only the stimuli were cognates. Examples of this type are: 

1. conclusion/end " 
cone lusi dn/ f in 

2. terror/ fear 
terror/espanto 

Partial Cognate * pairs tended to be morg or less difficult as 
a *f unction of the frequency of the English response word/ 
Thus. while the first pair above was about equally difficult' 
(.43/.46) the second pair was not (.52/. 79). Occasionally, 
the difficulty levels of such item pairs, was greatly dis-* . - v 
cprepamt as in the case of: * *• 

negotiated/bargained ( . 2lt) 

negociaron/comerciaran (.87), 

, In those item pairs where 0 cognates .were present ^| there 
was a tendency for finglish and Spanish item scores to be *>* 
approximately equal. . Since the total test^scQres for the 
-vocabulary subtest were not equal, we Wfere^interested in — - 
,conc!ucting~ an errot ^analysis on -those items where the discrep- 
ancy between English and Spanish item scores was larger. The 



:QJ» 67. 



approach, was first to identify the relevant English language 
items", tfcen to examine the frequency of response t& each of 
the distractors. * 

/ Two m4j° r pattern^ of wrong response -occurred. In the 
case that a student? dik not know the right answer and there 
were- no clues to such/an answer, responses tended to be ran- 
dom, giving a typical .response -proportion of close to .25 
for each -option; There were cases, however , /where the stu- 
dents went fbr on^ of the distractors in a strong way* In 
most of these cases, the. option chosen bore a strong physical 
resemblance to the underlined stimulus^ word. Examples of x 
such pairs are (proportion of response in parentheses) : 

* , 1# conclusion/comparison 

2. -adequate/adjusted " 

3. d$tta/dktes / 

4. incident/accident 
- 5. environment/investment 

6. > installment/implement 
..."^ 7. legitimate/legible 
8 . negotiated/negated 




That students were paying attention to physical similarity 
between stimulus words and distractors is attested by the 
fact that Random patterns of response\occurred«only on items 
where no distractor bore a physical v resemblance to the stimu- 
lus word. 

It may be useful, here to comment on response teridencies 
on those items where 'the creation of a test by translation 
resulted in some awkward translations.' Such items sometimes 
.caused problems for thQse subjects who took the CTBS Espanol« 
-*At other times, they,ca.used no^jfcrouble. Examples of the 
latter are: ' 



1. ' personalidad magnetica (*58) 
* . atractiva * : 

2. char la tranquilizadora (1.00)* 
caiman te , 

reaccion entusiasta (.63) 
-ansiosa 



Examples of . the former are:. * . . 

1." " discurso estimulante (-21) 

' i . % , \ dncitante 

. + ■ 1 ' j 1 . 

'2'. descripcion complicada (.00) 
eriredada 



There were other difficult Spanish .items where the cau§e 
oi; .the difficulty may have been more related to experience 
than to ianguage per se. For example, in one item, " jolly 
old man" is given as the stimulus and "merry" the correct 
answer. The ^conjunction of these particular words immedi- 
ately elicits the image of Santa Claus with whom the words 
"jolly", an^ "merry" are intimately associated. In the Span- 
ish version, the stimulus reads "un anciano jovial " and the 
correct answer "alegre." Though Santa Claus (San Nicolas) 
is a familiar figure in Latin ■ America, _the term "anciano ^ 
jovial" is not closely linked with him. Additionally, the 
wofd "jovial" is a low frequency word. The proportion of 
correct responses to the Spdnish version was .26, to the^ 
English version .22 indicating that the U.S. Christmas 
associations are not" known to our sample in English nor are 
• they mirrored in Spanish. By 'contrast, the English language 
norm sample demorfstrated a proportion of correct response 
of .77 on this item in English. 

Other errors^ may be directly related to culture differ- 
% ences.* For example, one item pair is given as: 

man of good char&cter7 qualities 0.29) 
un 'horabre de buena reputacidn/ fama (.21) 

One of the' response options on this item was "education/ 
educacidh." The proportion of responses to "education" was 
.62 and to "educacion" was .50. A Latin American Will 
quickly know why the students responded in this way. For % 
them there ^is an intimate link among character, reputation," 
and educacion . They responded correctly from the standpoint 
of their culture even though they got the item wrong. ; 

J We -may conclude the discussion 4 of response tendencies 
on the vocabulary subtest by presenting a list of guessing 
strategies that the. LEP students seem to have used on the 
CTBS/S when the correct answer was not surely known. These 
strategies are, of course, inferences. SGme of them are 
based on several items.. Some of them are based on fewer 
items. All of thenuare presented as hypotheses. 

H^: I£ the answer is not known but one 
distractor is within the' semantic / 
region, even if it is an antonym , 
choose it. 

If the answertls not known but there • 
is one distractor that is .frequently 
♦used as a modifier gf the stimulus, 
'word, choose it. 

H3: If the answer is not -known but there 
is one distractor that experiential ly 
-occurs in conjunction with the stim- 
ulus (e.g., aqcident , damage ) ,• choose 
it. 



H 4 : If * the, answer is not known but there 
is ,a didtractor whose orthographic 
;form is^similar to that of the stim- 
ulus word, qhoose it. 

H5: If the answer i^_jaotJyiownjDut one 

of t&e distractors- is a cognate (or * 
false 'cognate) of the stimulus word , 
. p choose it # ' 

&$: If the answer is not known but one of 
, the distractors is known to me$tn the- 
same as 'the whole item stem, chdose it* 

H7: If^the answer is not known and none of 
the ""clues" listed in the first six 
hypotheses can be founji, guess blind. 

We can now move on to a consideration of findings oh . 
the Reading Comprehension subtest. The -first step in the 
•analysis was to look for an effect due to language f of the 
test. A^ visual inspection of Tables 12 and 16, reveals that 
the English language version of the items was typically more 
difficult than the Spanish language versioiv* This difference 
was examined statistically and* was* found to.be significant. 
The MANOVA is presented later. 

In order to learn as much more as possible, the it£m "V 
difficulties wer,e organized by reading selection. Visual 
inspection of the arrays revealed that there were systematic 
differences in difficulty as a function of reading selection. 
Interestingly, the two most difficult selections on Level ,1 
were the two dealing with* scientific subjects, one with the 
invention of at phonograph and the other with satellites. It 
is interesting ^to speculate that the difficulty may be attri- 
bute4 tb lack of regular contact with "the world of science 
in the home countries. At the same time, 'it ipust be admitted 
that we are finable to separate such a hypothesized cause from 
other possible causes rooted in language itself. -'The nature 
of the passages in Level 3 made it impossible to check the 
hypothesis. It was found, however, that in Level 3 also, 
there were clear differences in difficulty as a function of 
reading' selection. ** % 

*» 

This analysis led to an, even more interesting observa- 
tion. This was the nature of the distribution of item dif- 
ficulties within reading selections. This led 9 to. speculation 
that there might be systematic variability' in item difficulty 
Is a function of type of task. To check this out, each item 
was categorized in terms**of the following task descriptions. 

1* locating details 

2. getting the ,main'idea 

3. establishing order of events 

4. drawing inferences. • - 



o. determining meanings of ^ords # 
and expressions * 

6. summarizing sub-sections 

7, selecting descriptive words : 

While a. visual inspection of item difficulties tended to 
suggest that items requiring location of detaiJL^were easiest 
there was such a confounding of task difficulty with language 
difficulty that it^ was decided 'to try to reduce some of 'the 
"noise 'Since .it hacl been noted tjiat £he' LEP students re^ ' 
sponding to the Spanish version earned lower item scores on 
some items than did FEP students responding to the English 
version f it seemed plausibly to assume that on these parti- 
cular- items , the LEP students wfere experiencing some diffi- • 
cirlty over and above any possible language difficulty. It 
was 'therefore decided to look at these; i^ems.as a special 
subset* An item was' selected for further investigation ii • 
the Spanish version/LEP student difficulty index w^is at least 
.16 lower than the English version/FEP' student index. .The 
selection of the criterion .16 was arbitrary ^ 

' > • , < 

At Level l f there were 13 items. out of 41 (items 42-4'5 
were, omitted from the analysis since the Spanish and English 
versions were based on different poemfe} that met the cri- * 
terion. The distribution of these' items by task type was as* 
follows: 

•-1. locating details (items 13 , .19** 36, 
- 39) 

2. getting the main idea (items 15, '20) . 

3 . establishing order* r>f event? « « 
(item 8) * \ : 

4i drawing inferenqes (items 12/ 27) . * 

5. determining meaning of words and 
expressions[ (items 18, 25, 32)* t 

6. summarizing sub-sections -(item 14) 

7. s selecting descriptive words (item 22) 

— *. , 

„ Note that "iocating, details" ■ accounts 'for 30 percent of' 
the, items in this 'subset while items requiring this skill 
m£ke vtp 45 percent of' all, items in( the subtest (items 42-45 
omitted). This may indicate that there are indeed * task," 
factors that create, the observed increased difficulty. 

The reader should not infer * from the above that the 
study' sample rt^ver had difficulty in locating details nor 
that they always had difficulty Aaking inferences, etc. . The 
.finding is merely that, on those items which LEP ^students 
found/ Excessively more difficult than did FEP studerit^, the 
tasks were most .likely to require more* cognitively "demanding 
skills. These findings may be interpfetable in /terms Glk/ 
Cummins ' recent', formulation where the difficulty of a given 
language task is a function of the interaction of, -two factors 



namely # its contextual- embeddedness and its cognitive demand- 
ingness (Cummins, 1981). We shall .discuss this hypothesis in 
greater detail in the discussion section of the report, 

. It would have been preferable if the generalizations 
about item difficulties as a function of the interaction of 
i linguistic and- task factors could have been checked on Level 
__3_ of the test*.-.- -This- was-not^os^sibleTsrince^item difficulties 

• on that test were based on a mixed sixth and eighth grade 
study' sample. This altered item difficulties in the study 

* sample such that comparisons with the, norming sample were 
questionable* • ' 

t • * 4 

We turn now to an analysis of items on the Mathematics 
Concepts and Applications subtest/ (An analysis of items 
dn the Mathematics Computation subtest was not done since 
. item difficulties on the Spanish and English versions Were 
essentially equal.-. This could be attributed to- the fact that 
, language proficiency' plays no 'significant role in the latter 

* subtest) . 

r. '* 9 * 

. In contrast, language proficiency would intuitively seem 
to play an important role in responding to the Mathematics. 
Concepts and Applications subtest. As we shall see later, 
differences in subtest scores as a function of the language 
of the test were significant . at Level 1 only* though even on 
Level- 3, the absolute value of the mean on the English ver- 
sion was lower than on the Spanish version. 

m 

The analysis of item ^difficulties on the Concepts and 
Applications subtest was conducted in a manner similar to 
the analysis of item difficulties on' the Reading Comprehend 
. * sion subtest. 'The appropriateness of this analysis was sug- 
gested log the fact -""that on many ■ items, LEP students 'taking 
the Spanish version did as well as- FEP* students taking the 
English version.', On other items, the LEP students did sub- 
stantially more poorly. This suggested, once again, the 
study sample might be experiencing difficulty over and above 
that due to' language alon6. V 

Using the . 16- criterion, a subs.et of 23 Level 1 items ' 
was, selected fQr- special examination. The following list 
.gives the kinds of conceptual areas on which LEP- students 
did more poorly than FEP studehts regardless of the language 
of the 'test (unless 'otherwise indicated): 

1. place value 

2. reading time from a clock f;ace 

3. determining' the number of U.S. coins 
required to make, a given sum 

4. . working with the distributive and 

commutative properties of numbers 
. 5. distinguishing, between odd and even 
numbers (problem on English version 
\ • . only) 



i 



6. comparing, numbers of items' in two sets 

7. combining two geometric figures to 
* create a third figure* 

8. . determining calendar time 

9. deter mining. 1 ^ngthfh^rf^x^jeeSs^Tn 
fee t^ and inches .(most difficult in 

English}- • v 
10". determining unknowns in an equality 
Jll. determining the missing number in a 

number series 
12. computing values in~ word ^problems 

The finding for (12) was surprising. One might have 
guessed that there would be a strong effect Sue to language 
.in word problems. ' Such an effect did not appear. LEP stu- 
dents did equally poorly on these items regardless of the 
language of the test (only one exception to this .in eight 
items),. 

A similar analysis for Level 3 was not conducted because 
of the problem of combining data for sixth and. eighth graders, 



An Analysis of Effects • 

Separate but parallel analyses were conducted : '£ or Level 
1 and Level 3 of the CTBS. These analyses included an analy- 
sis of scale reliabilities as well as a multivariate and uni- 
variate analysis of variance. Two multivariate analyses were 
performed. First, "reading" was investigated using reading 
vocabulary and reading comprehension as dependent variables, 
and theft* "mathematics" was examined with mathematics compu- 
tation and' mathematics concept^and applications as dependent 
variables. The results of these analyses are presented in 
this section, while their ' interpretation is contained -in the 
subsequent section. 



— Reliability Analysis ... ~ , 

One of the important concerns for this study was the 
psychometric properties of . the- Eftgliish and Spanish versions.* 
of the CTBS. ^ These properties were investigated both empiri- 
cally ,and judgmentally, and- at the item and scale levels. 
The results o£"«the judgmental: and item analyses have been 
presented in a preceding section of* the, report. This section- 
discusses the result of. the analysis of scale properties'. 

Each of the' four test scales, used in this study was 
submitted to an analysis of its psychometric properties. The 
analysis was performed using the SPSS software routine en- 
titled "Reliability." This 'procedure provides a thorough" 
item and scale analysis t employing a variety o£ test score- 
models; In the present' case; we, used a traditional test 



theory model with coefficient alpha as, the measure of scale 
reliability, * This coefficient i*s a measure of internal con- 
sistency for the items of the scale and represents the average 
split-half reliability for all possible combina|fcLons of items. , 
The results from these analyses are reported in Tables 19-20. 
As mentioned previously , separate analyses were conducted for 
the English and Spanish versions of the CTBS. 




the Level 1 results has two important 
labilities for all four scales are 



An examination on 
features. First , the 

acceptably high (ranging fronP. 82 to ,96) and are very com- 
parable for; the two versions of the test. This would seem 
to4indicate that the scales are fairly homogeneous at this 
leTOl and across forms, Second f the performance on the Eng- 
lish version of the test is considerably lower than the Span- 
ish version. This discrepancy is largest for the subtests 
involving language {Vocabulary , Readingv Comprehension/ and 
Mathematics Concepts and Applications) and smallest in the * 
pure non-verbal subtests, that is, Mathematics Computation. 
Since comparable groups of students received the two forms 
of the test, this finding indicates a clear difficulty dif- 
ference in' the two forms. 



3$ 



The Level 3 results are 'substantially the same as those 
of Level 1. Again, the English version is considerably jnore 
difficult for the students. ■ However, the reliabilities for 
the Vocabulary and Reading Comprehension subtests on the 
English form are considerably lower than those on the Spanish 
form. The Reading Comprehension reliability of .55 is very 
low. ,This may mean that this subtest is not factorially pure 
and that more than a single ability is being measured here. 
Further, this lack of reliability will probably make it more 
difficult to find treatment effects at this level.- 

OveraJ 1 f ' +h£gg^^f?" J M^ ^f** striking in the difference 
in the il i m 1 MTTTyirvrl n $£ .the two test versions. This is 
.particularly true since the tests are direfct translations 
and the students are of comparable ability. 



, Analysis of Variance Results: Level 1 

The results o*f the analysis for Level 1 of the CTBS are 
presented in Table 21 for reading and Table 22 .for matheraa- • 

"tics. , Examination of these- results shows the presence' .of 
seyepal statistically significant sources of variance. In 
the. case of reading, and its component subtests, there was a 
very strong effect (p<.001) present for the test language. ' 
In addition, the method of test administration also was 

-statistically significant (p< .01) both univariately and 
multivar lately. Finally, the interaction of. test language 

.and test administration was also statistically significant 
(p< .05) . Examination of s the dell means contained in Table 
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TAB Lis 19: Scale Reliabilities, Averages 
i and Dispersions for All Students* 



Level 1 



Level 3 



to 


' English . 


Spanish 


English 


Spanish 


Vocabulary 


. X 


14.23 


23.15 


14.42 


23 t .80 


(40 . items) 


SD. 


6.84 . 


6.21 


5.70 


■ 8.00 






.84 


' .82 


.75 


.89 


Reading * 


X 


17.15 


24.28 


16.84 


23,40 


Comprehension 


SD- 


8.61 


9.81 


4.50 


8. -34 


(45 items) 


. 


.88 


.91 


.55 • 


.89 , 


Q 

, Math 


X 


3^.31 


37.75 


22.87 


25.30 


Computations 


' SD 


9.84 


11.45 


' 10.68 


9.99 


(48 items) 


cx 


.93 


.96 


..93 


.92 


Math Concepts 


X 


22.50 


29.50 


.L8J42- 


— tztSq"- 


(50 items) 


SD 


8.84 


.93 


— Tb"7l3 


9.68 
.90 • 



♦Treats omits as incorrect 



TABLE 20: Scale Reliabilities/ Averages and Dispersion 
•for Students Who Completed Each Scale Item 



< 




Level 


.1 , 


Level 3 

9 


English 


Spanish 


English 


Spanish 


Vocabulary 


X 


16.59 






24.23 


• 

15 27 


24 50 


(40 items) 


SD 


6.57 






5.22 ■ 


5.69 


8.11 




o< 


.'82 






• .75 

* 


1A 


90 


Reading 


X 


21.37 




■ 


31.56 


JLO • 7g 


97 Id 
& 1 • JL*± 


Comprehension 


SD 


8.08 






6.99 


. 4.53 


8.84 


(45 items) 


oC 


.84 






. .81 


..50 


.89 


Math 
* Computations 


■» 


37.32 
7.25 






41.00 






X 






25.90 
8=732" 


9.34* 


(48 items) 












.89 


.90 " '• " . 


Math Concepts 


X 


27.67 






33.12 


26.00 


25.88 


(50 items 


SD 


* 6.97 






8.15 


7.90 


9.71 




cx 


.80 






f.Ql 


.82 


.87 
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TABLE 21: ANOVA for Level One CTBS 

Reading Vocabulary Subtest 



Source 


DF 


ss 


F 


Pr "» P 


1 Testwiseness - 


1 . 


10,83 


d.31, 


0.5820 


Lancruaae ' u 


1 


1324-35 


> 

37 35 


n nnm 


TW* Language V*_J^ 


1 


159.18, 


* 4.49 


0.0371_ 


Administration 


1 


201.18 


5.67 • 


0.0195 


. TW* Administration 


0 


0.00 






Language* Administration 


1 


287.72* 


8.11 


0.0056 


TW* Language** Administration 

. Ei 


0 


0.00 







J 



ANOVA for Level' One CTBS Reading Comprehension Subtest 



Source 


DF 


ss 


F 


Pr ? F 




Testwiseness 


1 


223.53 

• 


3.47 


Q.0660 .. 




Language 


' 1 


874.15 


13.58 


0.0004. 




j TW* Language 


1 


135..25 


2.10 


o.isio 


K 


Administration 


1 


* 1128.72 


17". 54 


0.0001 




, TW* Administration 


0 


"0.00 








Language* Administration 


1 


335.38 


5.21 


0.0250 




— _TW* Language* Administration 


0 . 


0.00 




\ 


4 


' ■ . " ( 
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TABLE 22 : ANOVA for Level One CTBS 

Mathematics Computation Subtest 



Source 


DF 


ss 


p . 


'Pr T F 


/Testwiseness ' ' 


1 


" 548.12 


6.20 


0.0148 


Language , ' 


1 


170.23. 


1.92 


0.1691 


TW* Language f " ., >■ 


V. 


37.45 


0.42 


0.5171 ' 


Administration 


> 1 


^ 237.79 


2.69 


0.1040 ' 


TO* Administration - 


0 


0.00 






Language* Administration 


1 


412.33 


'4.66 


0.0338 ■ * 


~ TO*. Language* Administration 


0 


0.00 







ANOVA for Level One CTBS Mathematics Concepts and Applications Subtest 



Source 



DF 



SS 



Pr? F 



Testwiseness 
Language 
3W* Language 
Administration 
3W* Adn^wLstfation 
m I^nguat^ft Adminis tr a t ioA 
W* La^Hl 6 * Administration 



1 
1 
1 
1 

0 
1 
0 



40,79- 
679,77 

5.05 ' 
152.18 
0.00 
35.09 
0.00^* 



0.49 
8.10 
0.06 
1.81, 

0.42 



0.4878 • 
0.0056 
0.8060 
0.18i9 

0.5197 



24 , clearly shows hicjher ^levels of performance for students 
who received the Spanish version 6f the CTBS and who were 
administered the test, orally-. No statistically significant 
effects associated with the testwiseness training were iden- 
tified fyere, though the multivariate and univariate results 
.for reading comprehension were in a ^positive direction. 



The results fo£ mathematics were similar but npt quite 
.as dramatic as thdse for reading. Again, test language and 
the language- by ^administration^ interaction term were statis- 
tically significant at the multivariate , level. However , 
examination of the univariate results shows the language 
effects to be confined to the Mathematics Concepts Ap- 
plication subtest, -while the interaction term is significant 
only for the Mathematics Computation subtest. Of interest 
here, is the presence of a statistically significant (p < 
.05) effect associated with testwiseness training. This re- 
sult obtained both at the multivariate level and at the uni- 
variate level. Examination of the cell means in Tables 23-24 
shows that students who received testwiseness training tended 
to perform *sp mew hat higher on the Mathematics Computation 
subtest. 

Overall, these results suggest the preserve of a strong 5 
tes't language and test administration factor, particularly 
in the verbal areas. Additionally, there was some indication 
that testwiseness was influencing performance, though its 
strength was considerably less than that of the other factors 



Analysis of Variance Results: Level 3 

The results of the'Level 3, analysis are presented in 
Table 25 for the Reading subtest and in Table 26 for the 
mathematics subtest. Examination of these tables shows that 
the Level 3 results were statistically non-significant for 
the most part. The only significant effect was that of test 
language for the reading (multivariate) . and its "component 
subtests ♦ No other main efJEeqts or "interactions were sta- 
tistically significant. The results of the mathematics sub- 
tests produced no statistically significant results. Overall 
then, the Level '3 results were confined to a statistically 
significant test language effect for the reading subtests. 
These results correspond to that foun4 at Level 1, though 
other factors were also statistically significant at that 
lower level; 

A, general note of caution concerning these results for * 
both test levels needs to be made. Specifically, the rela- 
tively small sample size., and the low ^est performance 
(often approaching the chance level) probably makes these 
findings unstable, Thus they should be viewed as suggestive 
rather tHan definitive. « 



0. 79 

-65- • 



r 



TABLE 23: * Means for Level One CTBS 

Tested by Main Effects ANOVA 



* » 3 



Variable 


Condition 


Reading 
Vocabulary 


"~ 

-Reading 
Comp. 


Math 
^Gomp. 


Math 
Concepts 




No 


* 

■ ' 17.82 


20. §4 


40.59 


26.05 • 


TW 


(N=22) 










Trailing 


Yes 


19.64 


21.47 


- 36.39 


"26.53 




(N=66) 




















/ 




Spanish 


< 22.67 


24.06 


38.55 


28.90 ' 


Language 


(N=51) 










of Test 


English 


14.38 


17.41 < 


35.92 


22.97 




(N=*37) 












Oral 


20.97 


25.21 


38.88 


28 ..24 


Admin. 


(N=33) 










Mode 


Standard 


• . 18.11' 


18.89 


' 36.58 


25.31 




(N=55) 








y 






h 






* 

* - 
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TABllE 24: Means fpr Level One CTBS 
Tested by Three-Way ANOVA 



C611 a 



Language 
of Test 



Admin. 
Mode" 



Reading Reading 
Voc. Coriip. 



Math 
Comp. 



Math 
Concepts 



No 
TW 



Spanish 

0 

English 



Spanish 



-English 



Oral 
(N-20) 

Standard' 
(N=20) 



*Oral 
(N=13)- * 

Standard - 
(N=13) 



Oral 
(N=0) . 

Standard 
(N=ll) 



Oral ' 
(N=Q) 

Standard 
(N=ll) 



22.80 



23.50 



18,15 



10.31 



20.91 



26.20 * 42. £5 



22.35 33.25 



23.69 33.69 



- 10.62 34.92 



23.27 41.45 



14.73 



18.00 39.73 



31/45 



26.85 



23.31 



21.69 



28.00 



24.09 



V 



So 



» I 



TABLE 25: ANOVA for Level Three CTBS 
Reading Vocabulary Subtest 



, Source 



DF 



SS 



Testwiseness ; 
. Language 

* ■ ' 

TW* language , 1 

Adminis tratidn 

TW* Administration 

Language* Administration 

* TW* ^Language* Administration \ 





0.12 


0.00 


1 


1561.68 


•29.04 


1 


50.63 


0.94 
0.12 


1 


v 6.56 


. 1 


.3.36 


0.06 


1 


61.18 


1.14 


1 . 


86.10 


' 1.60 



0.9618 

0.0001 

0.3355 

0-.7280 

0,8034. 

0.2901 

0.2103 



ANOVA for Level Three CTBS Reading Comprehension Subtest 



Source . j ^ 


DF 


SS • 


F 


Jf7F 


Testjwiseness * 


i 

l 


3.46 


o.oe' 


0.7998 


Language 


l 


586 . 62 


• 11.00 


. 0.0015 


TW* Language 


l 


0.05 


• 0.00 ■ 


0.9760 


Administration 

• * 


l . 


, SL^T.84 


1.97 


0.1656 ' 


TW* Administration 


1 L 


10.70 ' 


0\20 


0.6556- 


Language* Adminis traction- 


1 


3.86 


0.07 


0.7886 


TW* Language* Administration 


1 ' 


79.54 


i:4*9> 


0..2284 



r 



' ■.' Mdi 82 



•I 



TABLE 26: ANOVA for Level Three CTBS 

Mathematics Commutation Subtest V 



; 

• i — 


• 










•j- - - - , — 




so 


r 


■ Pr ? F 


Testwiseness . Ci 


i 


- — *— 

0.15 


0.00 


v 0. 


9718 , 


x Language' . 


i 


11.82° 


-0.10 


0. 


7532 


^3^* "jjahguage 


i 


0.32 


0.00 


0. 


9585 


Administration * 


r 

r 


0.79 


0.01 


0- 


9353 


\ TW* Administration 


1 - 


20.95 


, 0.18 


0*. 


6757 


Language* Administration * 


\ 

1 ' 


■8". 63 


' 0.07 ' 


0. 


7882 


» •* « 
TW* Language* Administration 




171.43 - 


- 1.45 


0. 


2337 



ANOVA for Level Three -CTBS Mathematics ^Concepts and Applications Subtest 

\ « ... 





/Vsdurce \ 0 v 


DF 


SS 




Pr >. F <5 


Testwi^eness * 


1- t 


7.87 


0.07 


0,^96 


Lariguage • t . 


1 


178.15 


1.63 


0.2086 


*TW* Language 


1 


72.20 


0:66 


0'.4199 


Administration - - - 


1 


146). 29 


1.34 


v 0.2521 


TW* Administration 


1 


•'/•0.84 


» OlOl 


\ 0.9306 , 


Language*. Administration 


" 1 , 


). 6 « 18 


0.06 


O.Slfb- 


TW* Language* Administration 


1 . 


258.56 i 


2.36 


0.1293' , 



\ 



I- 

i 



i 



7f>r; 



S3. 



TABLE 27: Means foy Level Three CTBS 
Tested by Main Effects ANOVA 



Variable 


Condition 


Readirig 
.Vocabulary 


Reading 
Comp. 


Math 
Comp. 


Math 
Concepts 


Testwi^aness 
Tracing * 


V /• o 

No 

(N=32) 
Yes 
(N=41) 


19.25 

id. 58 ; 


19.84 
v 19.51° 


23.59 
23.00 

\ 


19.38° 

18^90, 

\ \ 

\ 

\ 


Language 
of test - - 

t 
• 


Spanish 
(N=36) 
English 
(N=37) 


; 

. * 14.-22 
* 


22.61 
• 16.78 , 


23.31 
23.22 


20*33 
17.92 


Admin. 
Mode 


* Oral 
(N=32) 
Standard 
(N=41) 


18.20 

» 

• * v 


21.28 
18.39' 


23.38 
23.17 

• 


20.9-7 
17.66 

»- 



J* 



4 W> 



fffb 8 4 



\ 






o « 




f 




i 
** 







TABLE 28: . Means' for Level Three CTBS 
Tested by Three-Way ANOVA 



Cell 



Ttf 
Training 



Language 
of Test 



Admin, 
4 , Mode 



Readi njj Read i ng Math Math 
Voc . Comp • Comp . Concepts 



TO- 



No 



Spanish 



English *' 



Spanish i 



r 



English 



Oral 
(N=ll) 

Standard 
(N=8) 



® Oral 
"(N=ll) 

Standard 

(N-ll).* 



Oral 

Standard 
(N=ll) 



Oral 
(N=4) 

Standard 
{N^llT 



21.45 



24.63 



17.64 



12.27 



25.00 



24.45 



13.00 



13.18 



22.27 22.64 



23.25 " 24.25 



19.00 24.45 



14.55 21.00 



25.83 25.00 



20.73 22.36 



18.00 20.00 



16.36 -25^36 



1-9.27 



19.88 



21.45 



15 . 27. 



26.5 



18.45 



16.25 



17.64 
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DISCUSSION AND CONCLUSIONS 



\ • Effects x>£ Culture 

t ' 

. • It is often claimed that norm-referenced English lan- 
guage tests like, for example, the CTBS are culturally 
biased. This hypothesis is, sometimes advanced to explain 
the relatively lower perforinance of language minority stu- 
dents on 'such tests. 

While the claim may be true, it is seldom accompanied 
by a systematic definition of the key term "culture." Apart 
from such a. definition, the claim has* little explanatory 
value* That is, there is nothing in the claim [itself to % 
help one to identify and correct specific offending cultural 
features. Since the aim of this study is the improvement of 
estimates of . achievement in students with limited English 
proficiency, it was incumbent on us to pfbvide a definition 
of the^term "culture" that would allow us to (1)* evaluate 
the validity of^ the claip and (2) to recommend some action 
to correct it. v We turn now to the presentation of such a 
definition. 

According to_Goodenough 7 ~ J ^uil^rce^cohsxsts of the vari- 
ous standards for perceiving, believing, valuing* and doing, 
that * fa person] attributes to other persons as a result of 
•their actions and admonitiohs. By our 'definition of culture 
the standards that a 'person thus attributes to a particular 
set- of others are for him the culture of that set. (Cited 
in Gibsonv 1976, p. 9)." 

^ Vlith this definition, Gooclenough places the locus of 
culture "inside" the person. . It "is important to recognize 
this point since most popular conceptions place the locus of 
culture "outside" the person, -usually in the form .of modes 
of dress, diet, customs, language, *etc-. TJie naivete of this 
lattier view of culture can be perceived by considering the 
folliowirig example. Suppose an Anglo-American should decide 
to begin- to dress like a Mexican «(but what does a represen- 
tative Mexican wear?) , eat what Mexicans eat -(and. what does 
a representative Mexican eat?), behave like* a Mexican (and 
how; does a representative Mexican behave?), speak Spanish ' ^ 
(and what about Mexicans who speak Indian 4 dialects, German,* 
Chihese, etc.), etc. Would our Anglo-American have, become' 
culturally a Mexican? Clearly he. would not. Culturally, 
he would be. exactly :_what, he was befpre but now he would be 
masquerading in terms of his Mexican stereotypes , none of 
which have . anything at all to do with what makes a Mexican 
what he culturally is. In faqt, by our definition- of cul- 
ture/ the tferm "Mexican" has little to do with culture. 
Rather/ the term si?ttply 4es6ribes a person's nationality- 



For o our purposes, the value of this definition: of cul- , 
turelies in the assistance ^t gi^res in identifying examples 
of culture bias as well as in* making distinctions between 
culture bias per se and other noni-cultural features of a 
test which nonetheless systematically ; affect a student's 
ability to make correct responses. By the definition we 
have adopted, we shall say that an item is culturally biased 
if getting the correct answer requires that the examinee be 
competent in* a culture in which he/she is not competent "or , 
more precisely, be able ta function in terms of the standards 
for perceiving, "believing, valuing, and doing that define 
the -culture. 7 * \ - 

Using this .definition as our standard, we looked for 
and found little of culture' bias in- the CTBS. In the entire 
item analysis of Levels One and Three, only one 'ihram was 
found that might be said to be culturally biased. All other 
cases in which the study subjects showed a strong tendency 
to choose an answer other than the correct bne, factors 
other than culture seem to have been involved., We therefore 
find the manufacturer justified in the claim that CTBS. scores 
are affecte'd minimally by culture Jbias.- - — 



Effects* of. Language . 

Clearly (and not surprisingly); the language jof the test 
was a more powerful determiner of test scor^^s than was cul- 
ture. The students did much better on the Spanish version ^ 
than they did on* the English version. - 

'Parenthetically, the reader Should note that, we treat 
language albngside and independent of culture, not as an ele- 
ment of culture itself. This is done because -the definition 
of culture that has been adopted does not permit its inclu- 
sion. The correctness of this decision can be appreciated 
by considering £he~fact that groups having widely divergent 
cultures often speak the same language. While language may 
be used as at medium- for communicating or revealing culture, 
it cannot be identified as sl part of culture itself. 




Returning now to the discussion 1 of tiie_findj 
foifftd that the subtests_Can--be- rankedHBy difficull 
of their dependence on academic English* prof ici< 
Reading Vocabulary and Reading Comprehension suL^_ „.„ 
the most linguistically demanding and therefore thei most 
difficult, the Mathematics Concepts and Applications .silbtest 
was the next most linguistically demandyig an£ therefore -the 
next most difficult, and the Mathematics Computation subtest 
was the least linguistically demanding and therefore the J 
least' difficult In fact, with regard to the latter subtest, 
we could distii^uish no difference* in the level' of difficulty \ 
between the, English and Spani^jj, versions;' We conclude, 

• ' 87"' , ^ 



therefore, that for the rion-English-proficient student, th^ 
more ^ subtest depends on English language proficiency, the 
more difficult it' becomes. 

■ • As we pointed out earlier, this, finding only states the 
obvious. Its importance uies perhaps in (1) documenting the 
magnitude of an effect whose presence is known to every bi- 
lingual educator and (2) in leading us to ask further ques- 
tions about the specific nature .of the language difficulties 
that language minority students encounter when they take 
tests like the CTBS in English. 

We were led in our analysis by theoretical work done by 
Cummins. In his earlier theorizing, Cummins 1 research led 
him to distinguish between conversational skills (Basic 
Interpersonal Communication skills, i.e., BlCsjf and and aca- 
demic language ^cills (Cognitive-Academic Language Proficiency 
i.e., CALP) . H^M^sited distinct growth- curves for each of 
these two types 6T language proficiencies. BICS was Assumed 
to "develop rapidly from birth to age five, then to level off, 
reaching asymptote later in life. Growth in BICS was assumed 
to 'be a function of language acquisition (as opposed to lan- 
guage learning) . 

> 

In contrast, CALP was assumed to develop les* rapidly 
to about &ge fifteen, then to level off, reaching asymptote 
later in life,- Growth in CALP was assumed to be a function 
of learning (as opposed to language acquisition) (Cummins, 
1980) . * ' * 

The importance of the distinction -between BICS and CALP 
for this study lies in Cummins 1 assumptions about the rela- 
tion between BICS. and CALP in learners of* a second language. 
BICS in a second iahguagg is assumed to develop first:. * Stu- 
dents who have good 3ICS development are often quite 'profi- 
cient in tha second language. This, however, is frequently * 
misleading since full proficiency means that one is profi- 
cient not only in second JLangua'ge BICS but also in second 
language CALP. Students who lack proficiency in CALP, even 
though they are proficient in BICS, .may yet faiT tests iike 
the CTBS, which ^re tests of CALP* 

^Aj^ce that first formulation, Cummins has carried his 
theorflfcig further (Cummins, 1981) . In place of the two re- 
latively discrete language functions (BICS and CALP) , he 
posits a two dimensional surface on which the dimensions may 
be named, "level of cognitive demand" and "level of contextual 
embeotdedness . "^ LeVel of cognitive demand; is continuously 
distributed between tasks that* are cognitively undemanding at 
dne extreme to tasks that are cjognitively demanding o at the 
other. Level of contextual embeddedness is continuously' dis- 
tributed between tasks that are context-embedded at one ex- 
treme to J taskV that ^re context-reduced at the otfoer*. The 
crossing/ of these two continuums ^defines four quadrants. "* 
The- quadrants define four classes of communication as follows: 



1. Quadrant A: 
cognitively 
embedded 

2. Quadrant B: 
cognitively 

/ embedded 

3. Quadrant C: 
cognitively 
reduced 

4. Quadrant D: 
cognitively 
reduced 



communications that are, 
undemanding « and context- 
communications that are 
demanding and context- 
communications that are 
undemanding and context* 

aommuni cations that are 
demanding and context- 



Communications* in Quadrant A are characterized by the* 
**fact that "participants can actively negotiate meaning. . . 
and the language is supported by a wideyrange of paralinguis- 
tic and situational cues" on the one hand and by the fact 
_ that "the lingistic tools have become largely automatized 
. (mastered) and thus require little active cognitive involve- 
ment for appropriate performance on the other. " ~ These kinds 
o£ communications *are relatively .easy, -they include much of 
Cummins formerly called BICS. 
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Communications in Quadrant D are much more difficult. - 
They are characterized by heavy reliance "on linguistic 
cues to meaning" on the one hand and by the fact that the 
linguistic tools have not, become automatized and thus re- 
quire actdve cognitive involvement on the other. Character- 
istics of communications in Quadrants B and C may be in- 
' f erred- "from what has been^Sfeid about Quadrants A and D. 

*h - v 

We have taken some time to describe Cummins 1 work be- / 
cause we found it useful first in suggesting an approach to 
itein analysis and second in interpreting the results. 

' / / 

With respect to the CTBS , it seemed useful to think, of ' 
all- communications in the battery as falling in Quadrant's C 
and* D since the test format provides no- opportunity for ne- 
gotiation of meaning, no .parailinguistic or situational cues' 
and r> consequently forces a. §5tal dependence on linguistic • 
cues' to meaning. For a student with limited English profi- 
ciency* sugh flasks in English must n^cess^ily be difficult. 

Even so ,\£rpmiiis ' model would predict that such 'tasks 
would vary in difficulty as a function of the other dimension' 
* of the model, that is, as a function of cognitive deiuariding- 
ness. We Relieve that the item analysis data, are consistent , 
with the prediction 4£hough. certain non- linguistic features 
that are not a y part of the 'mode],, almost certainly contri- 
buted to. variations in item difficulty) . • 4 

We 'may cbnclude<this part .of " the discussion by saying 
that 'in all probability, a- major part of the difference in 
difficulty* between the English and /Spanish versions of the' 

1 ' . * I i ■ / 
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CTBS for the students in this study lay in their lesser 
ability to handle in English the contekt reduction that is 
an integral part of the battery. 



Effects of Providing Oral Assistance 

. ^ Some of the students in this study were given an oral 
administration of the Reading Vocabulary , the Reading Comr 
prehension , and the Mathematics Concepts and -Applications^ 
subtests. An analysis of the effects of this treatment re- - 
vealed no main effects. However , administration procedures 
did interact with test language f thus providing a signifi- 
cant advantage for students who received the oral adminis- 
, tration procedure in combination with the English language 
version. v This effect occurred on the Level One Reading* 
Vocabulary and Reading Comprehension subtests. 

This interaction is easily comprehensible since one 
would expect tha1afl»he target population would* be less profi- 
cient in reading English than in reading Spanish. The -fact 
that it occurred with fourth grade students but not with 
sixth -and eighth grade students may indicate that among sixth 
and eighth grade students , though their overall proficiency 
with context-reduced English- was poor, their reading skills 
and aural skills were a1b/zmt equal for this^ kind of English 
while for the* fourth grade students , ✓their aural skills 
were ahead of their reacting skills for this kind of English. 
,Thus, having items rea/a to them orally helped them to get 
some items right "thai: they would have missed otherwise. 

J * ' 

It should be noted in passing that, because there .were 
no fourth grade students in the treatment conditions defined 
as "no testwiseness/Spanish version/ oifel administration" 
and "no testwisene'ss/English version/oral administration, " 
the language by administration interaction could be tested 
only for students who had receivedL testwiseness training. - If 
one so" restricted- the analysis of Level Three data in exactly 
the same^ way,- the interaction may have been significant at 
Level Three also. The reader* can satisfy himself /herself 
on this by checking that portion of Table 28 dealing with 
students who had -had testwiseness training. The differences 
were clearly in favor of students who had received the Eng- 
lish version" and the oral administration. This seems to 
point to a nascjant possibility that the oral administration 
works best for students 'receiving the English version if 
they have also* had testwiseness training. The direction of 
the data favor such a hypothesis but at Level 3, the three- 
way .interaction wa^not-SjLgnifxcant. . 

/ • 4 / 

We conclude-, orf the basis of these considerations that 
those students' /whose. English oral skills are' ahead o£ltheir 
English reading skills may get more items correct if the> 



interns are read to them. Such a test naturally ceases to be 
a test of Reading and /becomes a test of ^ability to <pompre- 
hend orally-presented context-reduced English. This finding 
might. have important implications for diagnostic procedures 
in bilingual special education. The sampling limitations 
in this study prevent us from beingT more confident on this 
issue. 



Effects of Testwiseness Training 



Testwiseness training did not have the power that was 
anticipated. Since the training was directly relevant to 
the tasks required by the test and since the/ examinees who 
took the training appear not to have benefitted from it. we 
must assume either that all tlpfe examinees alread y possessed 
the relevant ifcills or that none of them aicH if the latter 
was the case ' (and it appears to have been), then the test!* £ 
wiseness training procedures and materials failed to 
% accomplish their objectives. Assuming that this is true, it 
/ is probably correct to assume that the failure was due tq 
insufficient amounts of practice on the skills referenced in 
the training materials. It would appear, therefore , that 1 
any re-work of those materials should incorporate more prac- \ 
tice exercises. Of course/ it would be eyen more effective 
~Tf teachers would incorporate these types/ of tasks into 
mainstream instruction. Students would then already be well- 
prepared to do tasks of those types , thus obviating the need 
for some parts of the testwiseness training materials. 

Other modifications in the testwiseness^training mater- 
ials were suggested by the item analyses/ These analyses 
\ showed that the students Exhibited certain systematic re- 
espouse tendencies that were counterproductive. It should be 
/possible to take advantage of this^in formation , using it to ' 
'modify the testwiseness training procedure, including speci- 
fic instructional sequences designed to eliminate these 
^tendencies and to replace them with more productive ones. 
If this were done, there is a high probability that testwise- 
ness 'training would result in significant dif ferences«in, 
favor pf students receiving such training. 



Effects of Using a Translated Test 
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\ The CTBS Espanol is a translation of the CTBS, Form S 
ih/fenglish. The translation was done by educators frony*;the 
Norwalk-La Mirada (California) ' Unified School District^ Fo£ 
the manufacturer , the translation* was a fortuitous gijft since 
the nature o$ the market would probably have discouraged "the 
manufacturer ffom producing such a test on its own. f For the_ 
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field of bilingual education, it was equally fortuitous 5 since 
it has permitted bilingual program personnel to document 
learning that might otherwise have gone undocumented, 

* 

Nevertheless, the creation of a Spanish language achieve- 
ment test by translation brought certain difficulties with 
it. In defense of the translators, it must be recognized 
that the items which might iiave created the most difficulty -- 
in translation were'SiQt translated at all. They were re- 
placed by new items written in. Spanish only. Thus, it is 
incorrect to say that the CTBS Espanol is totally a^ trans- 
lation from English, 

Having said that, certain problems remain. The most*T 
apparent problem might be called the problem o'f awkwardness . 
The translation from English to Spanish sometimes resulted 
in the use of Spanish expressions which are nofc^ gastizcu - — 
that is, expressions thaif would hardly ,* if ever, appealr in 
common written Spanish and, as a consequence, have an awk- 
ward feel to them, 

A second: problem has to do with word frequency . It 
was the .impression of the socio-linguist who contributed to 
the item analysis * that English words having a relatively i 
high frequency were translated using Spanish .words with a . 
relatively low frequency. Since it was outside the scope of 
this study to check relative word frequencies,, it remains as 
a hypothecs. Nevertheless, the principal investigator and 
the soeio-xineuist both feel strongly that a careful investi- 
gation would Support the hypothesis, -If the hypothesis were 
confirmed, there -would be good grounds for claiming that the. 
Spanish version- is more difficult than the English* version/ 

A third problem appeared in Level B of the test, a' 
level that was not used in the conduct of this s;tudy. In^ 
the Word Recognition I subtest, distractors weire creajbed us- 
ing both meaningful words and non-sense words. The Spanish 
versioh never used non-sense words. This is a non-parallel 
feature of the two versions which unwittingly* introduced 
phonetic "traps" in the English versiofi. For example in. 
Item Dl of the Word Recognition II subtest, we find a pic- , 
ture of a foot. The Students are supposed to mark ,the word s 
which names that picture. The four options are futV foot , 
look , and fish . The correct answer is. of cour^fe the second 
option. The three distractors all .look reasonable as long 
as we ate thinking in terms of a student population who. have 
English as the primary^ language. As soon, however, as we 
come to students with a primary language of Spanish we -find ' 
that the first distractor may become almost irresistably 
attractive. - This— is-s imply because of the phonetic character 
of the Spanish language and the pronunciation 1 of the word 
. fut* as in, for example ^Jbhe woird* futbol . Any Spanish speak-^ 
ing student who is unfamiytiar, with the intricacies of tijte 
English vowel system is likely to be irresistably attracted 



-78-. 




to fut and therefore to get a wrong answer. A similar kind 
of problem is found in Item 4 .of the same subtest. Here we 
have a picture of a hand. Tfie "response options are hand, 0 
han, have , bunt . For English speakers, the answer is clear. 
For Spanish speakers, it is. not at all clear since in Span- 
ish no^word ever ends in nd. It is known from experience 
.that: Spanish speakers' who are pronouncing English words of 
this type almost inevitably drop the final d. It is pro- 
bably safe to assume that the r^tson for dropping* the d is^ 
that they simply do not hear it. jGiven this . situation, the 
Spanish speaker may be almost irresistably drawn to the op- 
tion han which is an incorrect answer, ° Numerous other pro- 
blems of a^ similar nature can be identified. 

> r 

A fourth problem has to do with the possibility that 
two words that are dictionary ^ translations jof one another,- 
say house and casa , are psychologically and semantically 
distinct. Exi stin g s tu dies—of the. associative struct\ire:s""T ^ 
of such words in balanced bilinguals have* shown that this 
is % , .often the case. ; , * 

* , , The import 'of these observations is that the CTBS, 
Form S and the -CTBS Espanol may not be parallel .at all. 
Their factor structures* ihay be quite different.-. Even so, 
the CTBS Espanol represents/ a careful piece of work. In the 
absence of a non-translate^ Spanish language test of equal 
quality, the use of the CTBS Espanol can be highly beneficial 



Effects of Unfamiliar Item Content 

j ' . - . • 

*</ * " Being born and raised in one pl^ce automatically nteans 
^that one is likely to becpme famil^a^ with the environment 
' associated with that place, it also means that one is un- \ 
" likely tdjbecbme| familiar, wi£h the important features of „ 
environments with which one has had no previous direct ex- 
perience', 'if the .features of the unknown environments are 
incorporated into the content of test items, they are likely 
ta result in negative bias for those examinees for whom the 
item content is unfamiliar. This kirid of difficulty*, seems 
to have occurred with a few items. For example,- some stu- 
dents had difficulties with items that referenced U.S. coins. 
This is understandable fp^ the ^pop^lation in-question. Such 
questions cTearly favor those examinees who have handled such 
coins on a. regular basis # " 

It is impossible to tell from the data ^produced by ,this 
sutdy the number* of iiems oVer which 'the* examinees stumbled 
simply* because they had had 'no' prior experience with the 
things referenced 'in the te£t items. There may 'have been 
many *such items. ' 



Effects of Prior Training 



There was a significant number of items that were 
equally hard in both English and Spanish. On these same 
items , English-speaking students in the norm-sample did 
much better* In these cases ,' given that neither language 
nor culture can be invoked as an explanation for their 
comparatively poor performance, one must assume that in v 
their prior schooling , these students simply did learn 
how to solve these kinds of problems. It is therefore in- 
cumbent on teachers of these students to take note of the 
classes of tasks which the students find hard 'in either 
language and to provide them with specific appropriate in- 
struction ' and practice, ■ 



Limitations of the Study 

' v 

The greatest limitation of , the study -lies in the sample 
size. It may be that in a larger sample study, some of ,the 
non-significant effects would have been significant* 

A second limitation seems to have been the inadequate 
numbers of practice, exercises* in the testwiseness training 
materials that could lead the students to the application of 
those same skills in ^the test-taking -situation. To correct 
this situation^ it would* be necessary to add more practice 
exercises.* and to expand accordingly the number of training 
s # essioi>s # 
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TO THE TEACHER 



j^he reason for this program * > ^ 

"Training in Testwiseness" was developed as part of a study 
entitled "Improving Estimates of Achievement in Students 
with* Limited English Proficiency.": The study was funded un- 
der a cost-sharing grant awarded to Applied Social Research 
by the National Institute of education. It was dedicated to. 
finding some ways to help limited English proficient (LEP) 
students to earn scores that would more accurately reflect 
their- true levels .of achievement. ' Anyone wrfo would like to 
have a copy of the study should ask for it by name. 'Call or 

mail your request to: ^ 

* * * 

Applied Social Research 
, 15219 -Sunset Boulevard, Suite 201 
Pacific- Palisades, CA *90272 
(213) 454-8464 - (213) 459-4264 



The goals and objectives of this program 

Many LEP students come to this country not knowing how to 
take tests like the Comprehensive Tests* of Bajfifi Skills 
(CTBS) .' They have not learned the strategies-, that knowledge- 
able .students use regularly to improve their scores. This, 
places the LEP students at^a disadvantage vis a vis these 
other studerits. * >I / 

* 4 

The goal of the "Training in ^Testwiseness" program rs to 
erase this disadvantage, to teach LEP students how to solve 
the kinds of problems ihat occur ~on the CTBS. Consistent 
with this goal, the program is designed to help students to 
reach the following obj'eetiyes: 

* The students can delineate some reasons why 
people have trouble scoring well on. tests. 

The students can d'escribe- some major features 
- of standardized tests. 

Cor 



The. students can mark their answer sheets 
test booklets) 'quickly and properly. . 



£ahd the importance of pay- 
directions.- 



The students und 
ing attention tc/ te 

The students understand the directions on the 
Reading Vocabulary Test. 

The students will 'check their answers periodi- 
cally to det^fmine^whether they are being marked 
\ in the correct Spaces, • _ 
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The students will write -down the questions (on 
scratch paper) that they Han't answer quickly, 
or those wTich could use further ^consideration 
toaasure easy relocation. 

The students will go back & oyer the answers "to 
th,e problems to which they have some uncer- 
tainty if time remains. 

The students can eliminate answers known to be 
wrong and guess from the remaining choices if 
they do not know, the answer . . t r . 

The students can understand ^he directions on, 
the Reading Comprehension ^ Test. , 

The students can identify the six types of 
questions ~-a,sked on the, ReadXhg Comprehension 
Test and the best technique^ for ansyering 
these * questions . 

The students can understand the* directions oi> 
the Spelling test. ; , 

JThe students can -implement; the strategy for 
swering 'spelling .items qtaickly: 

Th^students can understand the directions 
for the Language Mechanics test . • 

The stua^nts can implement the " strategy ^f or • 
answQringN^anguage Mechanics -iterfls quickly. 

The students^&an understand the directions^ on 
the Language jSxpression- test. 

The students can. implement the strategies for 
answering pfte Language Expression iteifts* 

The students can under&tand the directions 
for the," Mathematics- Computation Test- 

The students will put scratch -.paper , if used, 
under or next to the problem in the booklet ' 
and not copy the whole problem, t>nta the / ' . . 
scratch paper. > " • - % 

The students will' work those- problem^ first 
which *caxi be don^ easily and quiqkly. 1 < . 

The stucfents can understand the directions, 
for the Mathematics Concepts and Applications 
Test. ^ • 

The students* understand. that word groblems reh 
quire careful, reading and h^rd words usually, 
can't be^ skipped because . th^ir meaning is im~' 
portarit f or . understanding the problem. 

The 6tudent^f«id — stah'd .directions for -"Refer-^ 
ence SkilJfsTest.* * * 4 
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Professional ethics demand, that we not use for practibe any 
of the problems that are found on -the CTBS. That would be * 
teaching th^ test * One should, distinguish, however, between \. 
teaching the pest'^and teaching to the t^st. Teaching to the 
test involves inferring the objective underlying a giveff 
problem, then" constructing items for practice t^at have the 
same form ,as those that occur, on the test. Use of such'items 
teaches students how to solve problems l^ke those, thatr are 
found^on the test. ' That is something tha,t- is ethical, legal, 
and desirable^ The •practice problems in the present program 
were developed- consistent with that principle. 

J± is suggested that - 6ne go even further, presenting the 
pitectice materials in exactly fc the same^ format as the format 
of the test. This might; include not only the layout of the 
problems .on the =p^ge but also 'the layout and use of separate 
answer sheets. .What 6i¥e wants to do is' jto reduce* to .zero 
the probability that a student will be intiifyidated or con- . 
fused by the stimulus and/or response format/ of the CTBS. % 
By doing this, one increases the ..likelihood that the student 
will be able to tell you what you wanted to know in the first 
'place, that is, whether or not s/he knows how to solve pro- 
blems of particular types. 



Materials • . . 

Th$ materials consist of this manual plus a full set of stu- 
dent practice materials. The practice materials" are currently 
available in both English and Spanish. / . 

The manual has five chapters, each chapter providing content 
for one or more sessions. Each chapter begins with a list 
of tfte chapter objectives. The remainder of each chapter is 
divided into sub-se^ions, each ^job-section corresponding to 
a single objective. mm Within each sub-section, there is first 
a list of the major ideas that'are to be developed in rela- 
tion to the\given objective. Following this list is a set of 
actiyities designed to help students to accomplish the objec- 
tive. Each activi-ty is described in sufficient .detail to 
permit the teacher, regardless of his/her training in tests 4 
and measurements * (or lack" thereof ) , to conduct the, activity 
successfully^ • * \ 



Planning . . > . - -- , 

'Q.. • Who should receive the training outlined 'in this manual? 

A. The training was piloted successfully "with fourthy six-tlj, 
and eighth grade stxiQents. Since the training given is x 
appropriate for use with CTBS Levels 1-4; the effective 
operating range is fourth grade through twelfth grade. - 4 
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While the- original tar.get population .was Spanish-speaking 
LE£ students , the' program would be appropriate for use' - 
/ ' with any English or Sp.anish speaker, who ne^ds assistance 
■ in learning 'how to 'deal 'effectively with the CTBS.' 

Q. .*When should .the training be given? 

A. It is best tQ give the. training such' that it is completed 
only shortly before "the administration of * the CTBS. 

Q. How should training be paced?* 

A. A variety of options' are possible running from "massed" 

practice to "spaced" practice. In onfe schdol, eighth 
^ gra'de students ' completed the entire training program in 

one. day (i.e., massed practice). # In another school, v • 
/ a fourth grade students completed the training in Iqur ses- 
< - sions spread over four <|ays ,(i.e., spaced practice). The 
actual *pace you follow wili depend on your sense of how 
much sustained attentioiPyour students are capable of * 
giving £nd how much 'practice' they nefed. It is recommended 
that, in mos*t cases r tjple training 1 be: spaced over 'several 
sessions on several 4 a Y s ». The' iiripoptant thing is that 
"the students get', enough practice to' actually master the 
objectives. ' 



Providing for additional practice > 

You' may feel that some students need jnore practice than is* 
provided by the practice exercises included in this program. 
In such a case, the given practice^xercises "provide a model 
for the .cbns1?ructio/i of additional exercises. 0 
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SESSION L. 
Objective 

c 

The students can delineate some reasons why 'people have 
trouble ^coring well on tests. • 

The students can describe some major features of standard- 
ized tests. f , 

The students can mark their answer sheets (or test booklets) 
quickly and properly. 

The students understand the importance of paying attention 
to test directions. . . • 



- Session One ^ 

Objective: The students pan delineate some reasons why 
people hay^ trouble scoring well on tests. 

A. General Ideas 

1/ Soon we will be taking a test called the Comprehensive 
Tests of Basic Skills , the CTBS for short. 

2. This test is given to students all over the country. 

*- » • 

3. I am: going to tell you about the kinds of questions 
you will' be. asked ,to ^answer on the test. Thiis will % 
help you understand 'exactly what you will have to 

> do before y6u take the -test. e . 

* *. » 

4* We also <will spend time learning about and practicing 
the kinds of skills , techniques , and tricks that will 
help you improve your .scores on the test. 

5. Mahy people do not do^w^ll- on tests , even when they 
really know the information, because of £ variety of 
reaspns. Some of these reasons 'are:, nervousness, 
not understanding. what the 'directions mean, working 
" too slbwly, marking the. answer in the wrong place, etc* 



1. m Ask students to *share with the class hov* they feel 

about taking' tests and the reason^ they feel this * 
way, This activrty-i-n tends to show students that 
many, if not most, people ha#e concerns about test- 
taking. * , 

2. Put a. list of reasohs on the chalkboard of why many 
people do not do well on tests (e.g., too nervous, 
don't understand direptions, work too slowly, etc.)* 

' . Ask^ students to supplement the list on the bo^rd 
with their own reasons.. Discuss eachi reason and 
ask students to think" about which reasons ma^ apply 
to them. TelX, the' students that you will be teaching , 
them strategies which should help them in overcoming 
many of these problems people have with taking tests, 



B. Activities 



r 



II. Objective: ^The students can describe some major features 
of standardized tests. * 

A. General Ideas 

1\ The test that you will be taking, the CTBS, is called 
, * a standardized test, * 
. » . < -=_.„< 

2. A standardized, test tests certain skills of, yours, 
like reading,- spelling, and math, and your score 

* tells you how well you c&n perform these skills 
compared to other people in the country your same 
age. ' . 

3. The CTBS, the test you will be taking, will, have 
questions on, it that you are^ not expected to know 

--Jthe answers to* You should hot be bothered by the 
fact that you do' not know the answers to all the 
questions on the test. • 

S f 

4. To do well on a standardized, test, you do not haVe 

to answer all of the questions correctly. 



5/ What is important is that you try to do your* best . 
and work' as quickly as possible. - 

6. Your sco£e on the CTBS^will not affect* your grade- 
in school but it will become a part of your permanent 
school record^ . ' 
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Activit ies . * • 

♦ ' * x 

This activity /should help students remember some of; 
the features o c f standardized tests* put ti\e follow- 
ing true-false questions on the : chalkboard. 

(a) Xpur score on a standardized te.st tells yoii how 
well you perform certain skills compared to 
other students, in the country* (true) 

(b) You are expected to know the answers to all the - 
questions on a standardized test., (false) 

(c) Your score on the CTBS will not affect your 
grade in school. (true) 

(d) .In order to receive a high score oh a standard- 
ized test, you ifiust answer all tfie questions 
correctly. (false) s > 

Have the students write the answers to these questions 
on a sheet of paper. Ask for student volunteers to 
tell whether ^ the statements on the chalkboard are 
true or false. Discuss. each answer with the class. 



2. * It may be helpful to cohduct a discussion with the. 
.class differentiating standardized tests from the 
rests they regularly take in class. The major dif- 
ferences that should' be emphasized are^ (#1) : 

•Regular plass tests 



Standardized tests 



(a) May include material 
never taught in . 
school . 

(b) Score shows how well 
one performed on the 
test compared to 
others in the . country. 

(c) Score does not affect 
gratte ip school. 

(d) You are not expected 
to know all the an- 
swers to .the questions 
on the test. 



(a) Usually only includes 
material taught by 
the teacher. 

(b) Score shows only how 
well orve performed 
on the test compared 
to others in the class, 

(c) ' Score does affect? 

grade in schoc^L. 

(d) You are 'expected to - 
know all. the answers 
to the questions, on : 
the test.* 



Tell the students to listen and read fc caref ully because 
you will be j giving them a "true/false" testr immediately 
(#2). * * 



III. Objective: , The students can mark their answer sheets 
quickly and properly.* m - & 

A. / General Ideas , 

-Note: 'Students taking CTBS Level 1 will be marking 
* theij: answers in the test booklet. Students 
taking* CTBS Level 3 will be using separate 
answer.. sheets. The way answer, choices should 
be marked is the same whether they are marked 
t in the test booklet or on a separate answer 
-sheet. 

* c 

1. Most standardized tests are scored by machine. The 
machine cannot te'll the difference between your 
answers and stray marks'. 



2. Mark 'only one answer for each question. 

> 

3. Make your marks da^k and clear. 

4. Make one heavy dot or- line in the center of your ' 
answer circle. It will be easier to do this if 
you use a dull pencil. . 

Do not ''spend a lot of time 'blackening in the ansJtr 
circle because it is important to work f^st on a ' 
test Like this. 

6. The whole answer circle does not have 'to be black- 
• ened in for the machine that scores these tests to 
pick it up. 

7; Do not make marks anywhere else but in the answer 
•circle. Accidental marks may cause an answer to 
be counted ^wrong. ^ 

•8. If you make a, mistake'- or want to change an answer, 
erase yo*ir first ariswer completely. If you lea^e 
a little bit of pencil mark when you erase the first 
answer you chose , the machine that marks "your paper 
9 will not be able to tell which answer ^ou want. ^ 
It will mark the question wrong. 

B.. . Activities * ^ 

1. Drajk a sample answer set on the chalkboard like the 
^following: , , 

■ . i. © © • ® .© 

Demonstrate to students how 'to fill in the answer" 
C circle and how a changed answer should be completely • 

erased. " 0 . 

y& * \ * ... ,_ 4 : ioe 



J2. Pass out th£ Practice Answer Sheets (#3) to the 

class so that, the students' can practice marking th6 
* , answer choices quickly and properly. On the Answer 
Sheets , there are 10 items and 4 answer choices for 
§ each* item. Call ou£ the number of each item, artd the 
' number, of the answer choice" students should mark foft 
each item* For example , question 1, circle 3; ques- ' 
tion 2, circle 5; ^question 3, circle 4; question 4, 
circle 8; question 5, circle 2; qjpstioh 6^ circle 6; 
question 7, circle D; question 8 , ^circle E; 'question 
9, circle 'A; questiQn 10 , circle G. Check * to see 
^that -students are marking their answers ^with quick 
'pencil strokes ana not spending time filling 'in the 
circle completely. 0 >/ 

3. This activity is designed so that students can 

-identify answer qircles that are improperly marked. 
Put the following on 'the chalkboard; • ■ 



1- « • ® ® -® ' " 

2. # '® . ®. ® ' ' • 

3. ®' ® © . ®. ^ - 

4. ® ® Jft ® • / 

Ask students to write down wl^at the problem is with 
the way the above answer 'Choices have^beeri marked. 
Have student volunteers describe to the class what 
they think the problem, is with the" way e'ach of the. 
ahswer choices has been 'marked. * * 



Objective: The students understand the impQrtance of paying 
attention to directions. 



A.- General Ideas 



v 

/o/rc 



1. It is extremely important to read each wojrdinthe 
directions very' care'fully and listen carefully 'to * 1 

. th^ directions given orally by the teacher. 

i ' > 

2. .The directicms tell you-aboufc the kinds, of questions 

you will be'answerirtg and the basis on which you 
should select your answer. 

3. If you r do not follow the directions you will most 
* likely lose many points. • 

4. D6 ^ not assume you know what to do without reading' 1 
. r the directions carefully and listening to the jperson 

giving thetedt. * • 
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5. If you do not understand the directions, ask the fc 
person giving the test to explain anything that is 
unclear,* - . * 

6. Keep the directions in mind. while taking the test., 

% • 

Activity . „' ' - . , .* 

This activity focuses on the importance of following 
directions • Distribute to the class the^ practice 
exercise labelled Directions (#4)* Tel,l the students 
that this Exercise is a special set of tasks designed 
to s.ee how good they are at following direction^ 
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'•' SESSION 2 
Objectives 

•The students understand 4 the directions on the Reading ^ 
Vocabulary Test. . ... • * # e 

A The students will check their answers periodically to deter- 
mine whether they are being' marked in the. correct spaces. 

The students will writedown the questions . (on scratch 
paper) that they^ can't answer quickly/ or those which could 
use further consideration , to assure easy relocation. 

The students will go back over the 6 answers to the problems 
to which they have SQme uncertainty if time remains. 

The students can eliminate answers known *td be wrong and 
guess^from the remaining, chorees if they do not know the 
answer . 

* * •* 
, 

Session Two * * 

(..• .— , - ■ 

Objective: /The students understand th$ directions on the 
Reading Vocabulary Test. 

A. General Ideas 

1. There are -eight 'different sections of the CTBS, each 
Of .them is called a test.' 

9 " 

2. We will go over the directions for each test so that 
you will understand what you are supposed to do. e 

3. Often people taking* tests will miss questions , not 
because they do not know, the answers/ but because 

'they do not uii4erstand what they are supposed, to <io. 

4 • ' on the lookout . f Qr the important words in the • A 
. directions which tell you the basis on which you 
should answer the^uestions (e.g.-, Choose the word 
that means the samg as Choose -the best answer". 

The?e are the m words you have to pay attention to ,and 
ask the teacher for clarification if you do hot 
understand what, they mean. ' "* 
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* , 5. The first section o'f'the CTBS isr called the Reading 

v # Vocabulary Test and the directions sayto "Choose the 
/ *V . word that 'means t£e same, or about t^he same f ~*as the 
- t underlined word*" We will discuss very specifically 
what these directions mean. * 

B . Activities % • • ? ^ , 

1. This activity helps students understand what directions 
intend 'when they say "Choose a wdtd that has the same, 

> or about the same, meaning as another word. Put the 

words "big" and "large"* on the 'chalkbpard. , T611 the 
students that these are words that have about the 
• same 4 meaning^ - This means that these words can be 
flsed -interchangeably in sentences. Have students 
. # ' generate sentences using the words "big" and "large." 

Put other words on the chalkboard (e.g., fast, cold) 
* % and ask students to use these words in -sentence^ and . 

, to think of other words that can be substituted for * 
^ them but where the" original meaning is retained. 

" v ' ■ . > * 

2. This activity helps students understand' what it means 
• ta select the word that ttias the closest meaning to 

the underlined word from a' gfcoup'pf faur wo>rds. Put 
the' following example on the chalkboard; 

tiny babies, 
-(a) sraay.! • 

,(b) weak t u ' t 

t icY helpless . 

(dj crying 

: c Tell the class that although all, the answer choices 
could be used before the word "babies" only one of 
the choices has almost the same meaning as the word 
"tiny." Ask the students to select the word that 
means about the same* thing as th§ word "tiny." Go 
^/ over the correct answer with the class explaining, 
if necessary, why the other choices are incorrect.' 
Another eicample that can be used for this exercise is: 

fresh strawberries' 
('a) frozen " 

(b) red . ■ - 

(c) ripe . ; . 

(d) just picked r- % \ ^ - 



Objective: The students will check their answers periodically 
to determine whether they are. being marked in the correct 
spaces. (Note: This objective is -only ^relevant when separate 
answer sheets are used.) 6 
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&~ -General Ideas: 



1. When there are qufe^tibns that you cannot answer 
quickly you shpuld write down, the. number of the 
question on scratch paper so that you can return 
to it if you have time, 

* > ■ 

2. If you do skip a question and plan to return to it 
later , you will have to remember to skip ^ space on 

your^ answer sheet* » 

* * «. 

3. Students sometimes forget to skip a space on their 
answer sheets and consequently mark their answers 
in the wrong spaces. When this happens, their 

. answers will be. counted as incorrect, 

4. There are strategies to help make sure you have 
marked your answers in the correct spaces, 

(a) Check frequently that the number of the question 
you are ^marking on your answer sheet corresponds 
to the question you are answering in .the test 
booklet, 

(b) Some people- fintf it. helpful to vocalize bhe v 
question number and the answer nunjber as they 
look from the test booklet to the answer-. sheet, . 



Objectives: gtudents will write down or> scratch paper the 
number of those items they cannot answer quickly , or those 
which could use further consideration , to assure easy 
relocation, •« v 

K The students will go t^ack over the answers to 
the problems- to which they have some uncertainty if time 
remains, 

A, . Gene/r al Ideas 

: / 

\ 1. Standardized* tests have hard and easy questions, 

2, Since standardized tests are timed, you. ha^ef to ^ 
work as quickly as possible, X 

3, If you spend too much time on a hard -question-, you 
^ may not have time to finish the test. And, some 

of tihe questions you did not have time to answer 
may ha v^, been very easy. . . 

4, It i's better to answer all th6 questions first that 
„ i you can answer- quickly^ These are' the questions. 

-you know the answer to or can make a quick guess. 
We will -learn how to make good guesses later. 



5. when- you come to a question that you feel you cannot 
answer quickly, write the number of that question 
down on your scratch paper. 

6. WhejP you come to the end of a section (or the end 
of" the test) and still have some time remaining, 

. you can- look at; the numbers on your scratch paper 
to fi'jid out which questions you need to go back and 

work on, . ' 

* • * 

7. You can also mar]£ down questions that you want to 
spend more tim6 thinking about. 

• * ^ * 

* 8'. I will be reminding you throughout the, lesson to 
check your_ answers, particularly those you are 
uncertain of, if you finish early. 

Objective: * The students can eliminate answers known) to be 
wrong and guess from the remaining choices if they d& not 
^cnow the answer. 

,The students know that they should guess when 
they do not know the correct answer. • . * 

A* General Ideas . 

' - ^1. On those questions where you don't know the correct 
answer, you should guess. at an answer rather than- 
leave it blank. m 

,« ^ • 

2.. Sometimes test directions will tell you not to guess 
because points will be subtracted for wrong answers. 
We will be discussing how to make good guesses so * , 
that it is always better to guess than to leave the - 
r question blank. 

-. *f&£ »v 

3. When you are not sure about the right a^xSwer you* will 
have to guess iji order to answer "the que«-bi<3n. 

* 

4.. The first £hing to do when guessing' is to decide 
which answers could not be right. • • • 4 

5. From the answers that are left, guess which one^you 
think might be 'right and mark that answer choice on 

• your answer sheet. 

6. Mark down the number. of the question you answered by 
guessing .so that if you have time 'at the end of the 
test, you can spend* time trying to .figure out the 
correct answter. . • * - * 

7. '-'Do not leave any questions* unanswered. 
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Activities 



1. Lead the students to consider the probabilities 
associated with chance guessing They should come * 
to understand that the chances for* guessing correctly 
improve dramatically as one is able to reduce the 
number; of options -among which one has to choose'. 

You might approach this by reminding them of the TV 
game, show format where the contestant must guess 
among several doors one of which hides a prize. If 
there "are four doors, then the probability of winning 
the prize is 1:4. or 25%. If we have inside informa- 
tion that aj-lows us to eliminate one door> the 
probability is improved to 1:3 or 33*2%. If we have 
additional information that allows us to eliminate 
another w door, the probability is 1:2 or 50%. 

2. The following Activity* is designed to help ^students 
understand how to eliminate wrong answers first. 
Tell students the answer choices are one, two, three, 
and four and you know that two and three are not the 

, v correct answers. Ask students which answers they 
should guess between (i.e. one and four). Continue 
givincf students problems like these. Tell them that 
■"when they are able to eliminate at least one answer . 
choice, it makes it more liktely they will choose the 
correct Answer. 

3. Put the following examples on the .chalkboard and ask 

^ the class to try, to eliminate incorrecT answer choices 
and guess at the correct answer. Have student volun- 
teers tell how' they can eliminate certain answer 
choices even if they are not sure of the correqt 
answer f 

.e.g., Give means the same^as 

(a) invest 

(b) go 

(c) contribute 

(d) ; steal 

e.g.*, Which Word is not silled correctly 
# (a) camel 

# 'tb) pig 

(c) # girafe 

(d) ' donkey 




4; Ask students to articuj 
questions on a test. 

(i.e., 1* Answer all queSTt±6ns first that can be 
answered quickly. ' > 

. 2.., Check periodically to make sure you are 
marking your -answers in the right place. 
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3. Write the numbers of the* questions on . 
scratch paper that require more, time to 
answer and return to these later. 

4. If you do not know an answer , guess . ) 

By articulating these rules , it will help students ■ 
remember them. 

Have the students do Practice Exercise #5a or #5b. 
Tell them that this is like the Reading Vocabulary 
Test but that it is designed to give them practice * 
in doing *sorae of the things you have been talking 
about. Go over the instructions qn the facing page. 
Then monitor their work, . 
Make sure that: 

a. th£y. are marking the easy items first , 

b. phey are marking their answers in the right 
" places . * 

c. they are writing the numbers of omitted items 
on scratch paper. * 

Once 'they have finished , correct the items but not by, 
merely giving the correct answer. Do so rather by 
eliminating incorrect answer s, thus illustrating the 
guessing procedures that you have been teaching. 

, Distribute the Practice Exercises — Reading Vocabulary 
•Test C#6a and #6b) to the students. The purpose of 
this activity is twofold:- 1) to make sure the students 
understand the test directions , and 2) to make sure 
the students periodically check their answers to 
determine if they are marked in the -correct spaces. 

Ask students to read the directions to themselves 
ind look over the sample item(s) . Then ksk a student 
volunteer to tell the* class what the directions mean. 
Emph£si2e to the class that the most important part 
.of the directions is the sentence that sa^jL^" Choose- - *~ 
the word that means the samfe, or about TiKesame , as 
the word with the line under it." 

Ask students to complete^the practice exercises. 
Remind students, of the following information: l)They 
must work as! quickly, as possible;- 2) If there are- 
items that they cannot answer quickly,- they should* 
write their numbers^ down and do -them last, after they 
have* completed the easier items; 3) They should check 
regularly to see whether they aire marking their 
answers in the correct spaces; 4) If 1 there. is time 
remaining, Jbhey should go back and. check their answers; 
and 5) If, tney do not know an answer., they should guess 

As the first students, complete the exercises, let the 
re.st of the class know they have only a minute left. 
Go over the /correct answers with the students. 
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c . SESSION 3 

* Obpeffiiyes \ ' 

* r ' ) " \.f ■ * 

The student^" can understand the directions on the Reading 
Comprehension Test. 4 

The students can identify the* six types of "questions* asked 
on %he. Reading Comprehension Test and the best te'chnique 
for answering these questions. f 

^Session Thrse, ^ 



Objective; The students can understand the dilfections 
on the Reading Comprehension test. 

A. Gerferal Ideas - 

; % ' . x 

1. The second section -of the CTBS is called Reading 
Comprehension. * 

2. # The Reading Comprehension section is designed "to 

show how well you understand what you read r <*t 

3. You must read a short selection (e.g., a 'story, . 
poem, etc.) and then answer , questions about it. 

4. On this test the directions say to- choosy the 
" best" answer. The "best" answer is the one £hat 
most Accurately^ describes what occurred in the 
selection. 1 4 - 

B . Activities 

1. ' Distribute an example of the type of item encoun- 

"j tered in the "Reading Comprehension", test (#7). Point 
* V to the reading selection and to the iteirr based on it. 

2. Tell students the following information. On the 
Reading Comprehension Test, they are to choose the * 
best answer for each question. The be~st answer is 
the one that . most acctir a tely~ answer s'^jthe question. 
Sometimes more than one answer seems like it could 

• be correct, 'but in the Reading Comprehension test, 
the best answer is the one that most accurately - 
represents what is contained in the reading 
selection. 

Using the example you have distributed, demonstrate. - 
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how several of the, answer choices could be a • 
correct (i.e., plausible) answer to the question, 
but; that only one accurately represents what was 
described in the selection.' , 



Objective: The students can identify the six* types of 
questions asked on the Reading Comprehension Test and the 
best technique for answering # these questions. 

A." General Ideas r 0 
= " «f . • 

1. * There are basically six different kinds of questions 

that -are asked on the Reading Comprehensibn Test. 

2. These six different kixids of questions ask. for tHte 
\ fallowing: 

(a) details 

e.g., What happened? Why .difl ; happen? 

' \ - „ W|ien did happen,?. Who did' _? 

(1^) order in which events occur ^ * 

e.g., What happened first? 

(c) meaning of .wbrds from' context 

e.g./ What does the word ' in line mean? 

(d) using descriptive words 'l?ro£erly 4 1 
er.g., Which word b^st describes _? 

. (e) making inferences 

i * e.g., What does ^ show? "what is . the most 

: likely ca l use of ^_? Why did _ probably 

happen? 
(f> main idea or purpose 

, e.g., What is this 'story m^hly about? What 
is -the best title? 0 

3. * "Answering *the different kinds of questions^ on the 
• • Reading Comprehension Test requires the use of 

different strategies..! 

4. It is generally a .good , idea to fir.st read the 
selection completely- through as^ quiekiy as you can. 

5. The|i ,try to answer as marty questions- as you ban. , , 

Crr^^Ej&^ydu do not know the answer , to a question . that . ; 
•asks ^out d^t^ils or the order in which events 
, occur 'go,. back only to that part of the stdrv that 
Will help you answer the question. , * 

\ — Quickly glance through the story unti^ you come-. 

. to the part that answers the question.. 
j, — Do not read the whole story a§ain. # ^% * 
'<•* — Do ' not spend time -trying to figure^out words 
;ybu do not know unless they are important for 
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'answering the question.. . 

— Detail arid . 6rdet questions can be answered even 
if you do not understand many aspects of the* 
selection. . 

In order to answer meaning ° of c words^ from context 
questions', yoil should locat^fche line(s) in the ' 
selection ^ere the word is -usecl. 
^: — Do not ' read the whole st6ry again." 

— Figure out which' Answer choice is closest in 
meaning to the* way- the word is uaed in the - - . 
selection. ' * * \ " « c ! 

— All of the answer choic/es ma^y be .correct uses 
of the word, but only one -answer choice depicts 
the ,way the word is usfed in the selection. , 

— Meaning of words f romlcontejct questions may be ' 
answered kiiowledgeably even if there are other 
aspects of the selection that' you do not i/lnder- 

" stand. * < 

Answering descriptive .words questions requites 
Understanding how petQple-'in the selection* feel , 
how things look, etc.. and choosing the ^nswer 
choice that^most accurately^ depicts this under- 
standing.- 

Descriptive words questions require a careful 
reading of that portion of the selection where 
the answei: is described. ^ ■» * % . 

— There is no need* H^o carefully reread the entire 
selection. <L ' v 

— The cotreqt answer chdiqe uses words that are 
different from 'those .used . in the selection, but. 
mean about the saigje. • J ~ 

Making inferences questions require you to draw 
conclusions based on what * you read* • 

— The answer will not directly appear in the 
^election. _ //-.^ * 

— Generally *you will^j^ve to have a, fairly .gocTd 
understanding of the selection. 

— The correct answer choice Is the one which is 
the most likely conclu%ipr^ based on "the relevant 
information in the 's^lecliion. 

Mainide^'or purpose questions .< require an under- \ 
standing of the poant of the 5 selection. 

The answer will not. dire'ptly appear in the* 
* ~ splectibn. , ' 

You will need to have-a good understanding of 

the entire selection, • 
--.However, you do not need to know the meaning of^ 

every word. • 



-15-* 117 - ^ ; 



B . < Activities * 1 

1. Distribute Practice Exercises — Reading Comprehension 
^T#8a and^#8b) ( . * Ask students to read the first select 
tion. GO over each question with them helping the 
-students determine what kind of information tfie ques- 
tion asks for and the type of strategy whi-ch .should 
~* be used.> Have students: determine which .answer 

choices are correct and go over the answers with them. 



tiei^vsl 



-Students then^shofild read the second 9 selection and 
mark the answer choices which. they believe are : 
correct. Go over correct answers with the class. 
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SESSION 4 , . 

^ Objectives 

< * 

The ^students can understand 'the, directions on the Spelling 
test. 

The students cai) implement the strategy for answering " 
'Spelling items quickly. 

The students can, undferstanS the directions for the Language 
Mechanics 'test. r ■ ° 

The- students can implement the strategy for answering 
Language Mechanics 'items quickly. 



The. students can 'understand the directions on the Language 
Expression test'. 

\ 

tfhe students can implement the strategies for answering 
the Language Expression items. , » - ' 



Session Four 

Objective: The students can, understand the directions on 
.the Spelling test* * ' ' 

A. General Ideas 



.1. This is CTBS ' Test 3 — Spellirig. 
* . 

„ 2. Each item is a. sentence with a word or worcjs under- 
. lined. 

* * * * 

> 

3. You must decide which wordy if any, is misspelled. 

• 9 

B*. Activity. 

Distribute the Practice Exercise — Spelling to 
the class* Go over the directions to this 'test 
frith the class using the sample itenl(s) as an' 
example of what the directions mean. 
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II. Objective: The studeri\ts can implement the strategy for 
answering spelling items quickly* 

A* General Ideas * -\ 

, - • * 

- 1. Look at the underlined wo*d(s) ^irst. 

2. If- you know the word(s) is spelled incorrectly, 
Slacken the correct space. There is no need to * 
"read the sentence. 

♦ 

*> 

' 3. If the word(s) looks like it" has been spelled 

correctly, read the sentence to see if the * correct 
word has been used £Ti the sentence. 

* S 4. The spelling errj/rs in the sentences are of two 
kinds: (a) Sometimes words that 'sound the same, 
but are spelled differently and have different m 
meanings -are used instead of the. correct word, 
(b) Sometimes words that are phonetically similar 
„ (i.e., sound, the same, but are spelled differently) 
to the correct woord aire used. These phonetically 
similar v words actually are not words at all. 

* B . . Activities ' % 

* I. Give students practice in selecting the correctly 
spelled word with pairs of phonetic equivalents. 
The following 0 are some phonetic equivalents to 
use for practice: after -aftre, cri - cry, 
candy - candey, snow - snoe, coat - cote, fil - fill 
Put"\these phonetic equivalents on the chalkboard 
and ask students to write down the one in eacft pair 
wtiichr is spelled correctly. Ask student volunteers 
to offer the correct answers. 

** ' ■ » 

2. Students^who are clear about the meanings of words 

*that sound the same but -are spelled differently 
(i.e., homonyms) should have an easier time with 
e the spelling tests.' Give students practice in 
determining the meanings of various homonyms. The 
following- are some examples, of homonyms to use for 
practice: blue - blew, there - .their,, too - two, 
no - know, by - buy. Put these* homonyms on the 
chalkboard. Ask 'students to write a sentence 
using each word and write the meaning of each 
word. Student volunteers can offer the correct 
answers. • , 

3. Have students finish the Spelling Practice Exercises 
Remind them to work quickly. They should first look 
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at the underlined ^word(s) without reading the 
sentence* If they know an underlined word is 
spelled incorrectly, there is no need to read the 
entire sentence. They should simply mark the cor- 
rect answer choice* If the Underlined word(s) 
seems to be spelled .correctly, tHe entire sentence 
should 4 be read to determine if *^he^ holtonyra. with the 
proper meaning- is used in the sentence* 

TelL students to remember to guess if they are not 
sure of the correct answer*- 



III. Objective: .The students* can understand the directions on 
,the language Mechanics test. 4 

A. General Ideas: Punctuation 
— : *E 



< 



1, 
2, 
3. 
4. 



We will -be going over the directions for the CTBS 
Language Mechanics Test. 

There, ;are two sections to this test, punctuation 
and capitalization. ' 

In -the 'punctuation section, you are to determine 
whether the sentence is punctuated "correctly . 

•If the sentence is punctuated correctly you are to * 
choose the answer choice "none" (for Level i) or 
"best as it is" (for Level III). 

If the sentence.. \s punctuated incorrectly , cl\oose 
the answer with* the punctuation that would correctly 
punctuate the sentence. 



B. Acti^fity 



Distribute the Practice Exercises — Language 
Mechanics to the students. After reading-Jar lis-* 
tening to the- directions on the punctuatioh section, 
have student Volunteers explain how the sample item 
should bl answered according to the directions. 
'After the sample item has been answered correctly, 

1 put* the sample item on the Chalkboard punctuatifig 
it correctly,, (i.e., He plays basketball^ -football, 
and baseball*) Ask students which' answer choice 
they should mark if tfce. item were writterv in this 

' way*' Go over the correct answer with the class 



Ci*e; ,, for Level I 
it is*). 



'none"; for Level III "best as 
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Objective: The student can implement the strategy .for 
answering Language Mechanics items quickly. 

A. General Ideas: Punctuation 



1. Dp not spend time trying to* f igure<out the meaning 
-of every word in the sentence, 

{ 2. Look for cues that indicate punctuation errors , 

(i.e., dates without commas, abbreviations without 
periods) ♦ ' 

/ 3. Even -if 'there are words you do not know, it is 

likely you will be abl'e to figure out whether the 
sentence is punctuated properly, 0 

4.' This same strategy will be used, on the Capitalization 
' section, only capitalization errors rather than, 
piinctuation errors are the focus, 

B. Activity 

Students should answer the punctuation items on the 
Practice Exercises ~ Language Mechanics implementing 
the strategy for answering the items quickly* Go 
over' 'the correct answers, 

C. . ^General Ideas: Capitalizations 

1* In the Capitalization section ycfu-are to decide 
vhere a capital letter is needed. 

2/ Each of the sentences in this secti6n is divided 
• into three parts • 



3. x If a word in- any part needs 1 a capital letter and 

isi^ t capitalized, - for Level I, fill in the Snswer 
space under the part where the capital is needed;- 
•s for Level III, fill in the space on your answer 
sheet that has the same numeral as the part where, 
the capital is heeded. 

4. Remind" students to use the same strategy for answer- 
ing these items a^ they did on th6 punctuation 
section, ' * 0 " • . 

D. Activities and Exercises 

• •* 

1. Distribute the capitalization items on the Practice 
Exercises -r Language Mechanics. Go- over the 
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directions for this section using the sample item 
*as an example. Have students complete exercises 
reminding them to look for capitalization errors 
before reading the sentence for understanding. 



Objectives: The students can understand the directions for 
the Language Expression Test. 

4 v - The students can 'implement the strategies for 

'answering the Language Expressiqn items ^ 

A* General Ideas „ 

- 1. The Language Expression Test shows how well you use 
• the English language to express yourself . f 

2.. There are six ^sections of the Language Expression 

Test^and the directions are different for each sec- 
^ tion. This means you have to pay careful attention. 

3. There are severer strategies for answering the 
various Language Expression items. 0 < 

(a) Some items call for you to choose the most 
grammatically correct use* of the English lan- 
guage. With these i£ems, it is a good idea to 
vocalize to yourself- the, item, and answer choices 
»so you can try to hear the answer ^Jfehat sounds 
correct, If you "remembeir the relevant grammar 
rule, that may help^you 4 select the correct answer . 
* (b) Sc>m6 items ask you to determine the^cprrect mean- 
ing of wofcis in context of select the most." 
' appropriate .word to fit a sentence. These items 
require you to understand the context in which 
a ward is used. ^ Read the passage and see what 
the .meaning" .of a* specified word is in that pas** 
sage. If you are to select the best word for 
a sentence, read the sentence carefully to determ- 
ining Which word fits the meaning of the sehtence 
most precisely, 
(c) Some items require you to determine the, proper 
order of sentences -as they should appear in a 
paragraph. - With Level I, students are to deter- 
mine which sentence should be first. With Level 
III* students are to decide the proper order 
for all the sentences. With these items, the 
. meaning of each item %a important for determin- £ 
ing the proper sequential order. Some words 
.(e.g., Then> But,- And) .which start sentences are 
cues that these sentences are not the first ones 
... in the paragraphs • 
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ii Activity 



Distribute the practice exercises to the class. Go 
over the directions and sample items for each, section. 
Once the students can articulate what the directions 
mean -for a section , tell tKem the strategy for answer- 
ing those items. Let students practice answering the 
questions using the strategy. Remind students to 
work quickly , and guess by eliminating incorrect 
answer choices- when they are not sure oi the correct 
answer . 



SESSION 5 
Objectives 

1. The students can understand the directions for the Mathematics 
Confutation- Test. 

2. The students will put scratch paper, if used, under or, next 

* to the problem in the booklet and not copy the whole problem 
onto the scratch paper. . ♦ . 

3. The students will Vork those probleras^first which can be done 
feasiiy and quickly.^ * c % < . * , 

4. The students can understand the directions for the Mathematics 
Concepts .and Applications- Test. 

5. The students understand that wor£ problems require careful 
reading and hard words usually 'can 1 1 be -skipped because 
their meaning is important for understanding the problem. 

6. The students understand directions for Reference Skills Test. 

S^ssioQ Five • 

t 

Iw Objective: The students can understand the directions on 

the Mathematics Computation Test. ; r 

- A. General Ideas 

* 1,. The dirfection£ on the Mathematics Computation Test 

are straightforward. 

***** 

2-. There are four sections to this test: .a&ditfbn, 
subtraction,, multiplication, and .division. 



3. Tt is important to keep^ in jnind .tb§ operation 
~r s (i.e., adcl, subtract, etc. ) required for each sec-' 

tion. Sometimes people forget • the^Sre working on 
^ % a subtraction section, for example^ and start adding 1 

This will' lead to a* wrong "answer . Check yourself to 
make sure you have performed the proper operation. 

• • * 4 

* * *• • 

» * £> «- , 

II. . 6b}ectiv§: . The students will -put scratch paper, if used, 
under or next to the problem in the booklet and not copy 



A. General Ideas 

1. ' There are many problems in^the Mathematics Computation 

section 'that you will not be able to figure out in 
your head* For this reason you^are allowed to use 
scratch paper, - 

2. When you use scratch paper, do not waste your time 
copying the entire problem down,.. The way to avoid 
having to copy the entire problem down is to put 
the scratch paper next to or under the problem you 
are working on in the test booklet. v 



Objective: The . students will work those problems first which 
can be done easily and quickly. 

A. General Ideas . » 

i. , The problems in m each section (i.e./ addition/ sub- 
traction/ etc.) "of the Mathematics j Computation Test 
; seem to increase in difficulty as the section" pro- 
• » gresses. 

2* Therefore/ it may be better to do those problem^ for 
. ,each* section first which can be done quickly and* ' 
easily and then go back and finish the rest. ^ ( 

B. Activities . * 

1. Put the following strategy reminder on *the chalkboard. 
Mathematics Computation 

Taj Check operation. : 

(b) Don't copy entire problem on scratch paper. 

(c) Work easy problems first. * 

Ask students to articulate what each reminder means. 

2. Distribute the Practice Exercises to the class. Have 
students complete the exercises keeping the 'strategy 
'reminders* in mind. ' , 9 



Objectives* The students can " tinder stand the directions on the 
Mathematics, Concepts* and Application Test. ' 

A. General Ideas * 

* -> 
1. 'The Mathematics Concepts and A|J£li'cation Test shows ^ 
how well you can use' math words and concepts and do 
* ' word problems y 



2. Each 'item asks a question and you are to sele<^ the 
answer choice that correctly answers the quesfion. 

v ... * 

- Objective 1 ^. The students can understand that the problems an 
this section are word problems and mus*t be read carefully. 

A. General Ideas \ 

/ r . 

1. Word problems must be read carefully. 

2. In these problems , most of the "hard words <cannot be 
skipped becaxise they are important in determining 
which answer choice is correct. 

» 

3. The mathematics in this section is usually not too 
difficult , btlt the jnathematiQs vocabulary may be 
problematic* . < 

4. If yQu do not understand all* the words in the ques- 
tion , try to determine what the important words must 
mean. Then eliminate the answer choices that seem 

. like they provide, the wrong answer , and choose the 
ansyer choice that, seems like it 'is correct. 

B. Activities * v * 

1. Distribute the Practice -Exercises to *the class. 
Explain to the class that these exercises will give . 
them an idea of the type of questions that will be 
on the Mathematics Concepts and Application Test. 
But/ also e:qplain that there will be many more 
problems than those which appear on the Practice 
Exercises. 

2. Go over ,the directions \^ith- the class using'the 
sample item as an example. Have students complete . 

, the exercises* Have 'student volunteers give the 
correct answers to the problems and share with the 
class how they decided on their answers. 

Objective: The students can understand the. directions for- 
the Reference ^Skills Test. 4 % „ * 

A. General Ideas 

1. The X terns in the Reference Skills se.ction are about 
finding information in the library and- in books. 

t 2. ~^¥<S& ; -e&^T to choose the .correct answer for each item. 
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